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Preface 


New to the 4th Edition 


The 4th edition of Introductory Digital Image Process- 
ing: A Remote Sensing Perspective provides up-to-date 
information on analytical methods used to analyze 
digital remote sensing data. The book is ideal for intro- 
ductory through graduate level university instruction 
in departments that routinely analyze airborne and/or 
satellite remote sensor data to solve geospatial prob- 
lems, including: geography, geology, marine science, 
forestry, anthropology, biology, soil science, agronomy, 
urban planning, and others. 


Introductory Digital Image Processing (4th edition) is 
now in full-color. This is important because many digi- 
tal image processing concepts are best described and 
demonstrated using a) color diagrams and illustra- 
tions, b) the original digital remote sensing displayed in 
color, and c) digital image processing-derived color im- 
ages and thematic maps. The reader can now examine 
the color illustrations and images at their appropriate 
location in the book rather than having to go to a spe- 
cial signature of color plates. 


The goal of this book has always been to take relatively 
sophisticated digital image processing methods and al- 
gorithms and make them as easy to understand as pos- 
sible for students and for remote sensing scientists. 
Therefore, the reader will notice that each chapter con- 
tains a substantial reference list that can be used by 
students and scientists as a starting place for their digi- 
tal image processing project or research. A new appen- 
dix provides sources of imagery and other geospatial 
information. Below is a summary of the major content 
and improvements in the fourth edition. 


Chapter 1: Remote Sensing and 
Digital Image Processing 


Greater emphasis is now placed on the importance of 
ground reference information that can be used to cali- 
brate remote sensor data and assess the accuracy of re- 
mote sensing-derived products such as thematic maps. 
The “Remote Sensing Process” has been updated to re- 
flect recent innovations in digital image processing. 
Greater emphasis is now placed on the use of remote 
sensing to solve local, high-spatial resolution problems 
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as well as for use in global climate change research. 
This chapter now includes detailed information about 
the increasing demand for people trained in remote 
sensing digital image processing. Information is pro- 
vided from a) the NRC (2013) Future U.S. Workforce 
for Geospatial Intelligence study, and b) U.S. Depart- 
ment of Labor Employment and Training Administra- 
tion (USDOLETA, 2014) data about the 39,900 
“Remote Sensing Scientists and Technologists” and 
“Remote Sensing Technicians” job openings projected 
from 2012—2022. Many of these geospatial occupations 
require training in remote sensing digital image pro- 
cessing. 


Chapter 2: Remote Sensing Data 
Collection 


This chapter provides information about historical, 
current, and projected sources of remotely sensed data. 
Detailed information about new and proposed satellite 
remote sensing systems (e.g., Astrium’s Pleiades and 
SPOT 6; DigitalGlobe’s GeoEye-1, GeoEye-2, World- 
View-1, WorldView-2, WorldView-3; India’s CartoSat 
and ResourceSat; Israel’s EROS A2; Korea’s KOMP- 
SAT; NASA’s Landsat 8; NOAA’s NPOESS; 
RapidEye, etc.) and airborne remote sensing systems 
(e.g., PICTOMETRY, Microsoft’s UltraCAM, Leica’s 
Airborne Digital System 80) are included in the fourth 
edition. Technical details about decommissioned (e.g., 
SPOT 1, 2; Landsat 5), degraded (e.g., Landsat 7 
ETM”) or failed (e.g., European Space Agency Envi- 
sat) sensor systems are provided. 


Chapter 3: Digital Image 
Processing Hardware and 
Software 


As expected, the computer hardware (e.g., CPUs, 
RAM, mass storage, digitization technology, displays, 
transfer/storage technology) and software [e.g., multi- 
spectral, hyperspectral, per-pixel, object-based image 
analysis (OBIA)] necessary to perform digital image 
processing have progressed significantly since the last 
edition. Improvements in computer hardware often 
used to perform digital image processing are discussed. 
The most important functions, characteristics and 


sources of the major digital image processing software 
are provided. 


Chapter 4: Image Quality 
Assessment and Statistical 
Evaluation 


Basic digital image processing mathematical notation 
is reviewed along with the significance of the histo- 
gram. The importance of metadata is introduced. Visu- 
al methods of assessing image quality are presented 
including three-dimensional representation. Univariate 
and multivariate methods of assessing the initial quali- 
ty of digital remote sensor data are refreshed. A new 
section on geostatistical analysis, autocorrelation, and 
kriging interpolation is provided. 


Chapter 5: Display Alternatives 
and Scientific Visualization 


New information is provided on: liquid crystal displays 
(LCD), image compression alternatives, color coordi- 
nate systems (RGB, Intensity-Hue-Saturation, and 
Chromaticity), the use of 8- and 24-bit color look-up 
tables, and new methods of merging (fusing) different 
types of imagery (e.g., Gram-Schmidt, regression Krig- 
ing). Additional information is provided about mea- 
suring distance, perimeter, shape, and polygon area 
using digital imagery. 


Chapter 6: Hectromagnetic 
Radiation Principles and 
Radiometric Correction 


Additional information is provided about electromag- 
netic radiation principles (e.g., Fraunhofer absorption 
features) and the spectral reflectance characteristics of 
selected natural and human-made materials. Updated 
information about the most important radiometric 
correction algorithms is provided, including: a) those 
that perform absolute radiometric correction (e.g., 
ACORN, FLAASH, QUAC, ATCOR, empirical line 
calibration) and, b) those that perform relative radio- 
metric correction (e.g., single and multiple-date image 
normalization). 


Chapter 7: Geometric Correction 


Traditional as well as improved methods of image-to- 
map rectification and image-to-image registration are 
provided. In addition, this edition contains an expand- 
ed discussion on developable surfaces and the proper- 
ties and advantages/disadvantages of several of the 
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most heavily used cylindrical, azimuthal, and conical 
map projections. MODIS satellite imagery is projected 
using selected map projections (e.g., Mercator, Lam- 
bert Azimuthal Equal-area). The image mosaicking 
section contains new examples and demonstrates the 
characteristics of the USGS annual mosaic of Landsat 
ETM” data (i.e., the WELD: Web-enabled Landsat 
Data project). 


Chapter 8: Image Enhancement 


The image magnification and reduction sections are re- 
vised. In addition, the following image enhancement 
techniques are updated: band ratioing, neighborhood 
raster operations, spatial convolution filtering and 
edge enhancement, frequency filtering, texture extrac- 
tion, and Principal Components Analysis (PCA). The 
vegetation indices (VI) section has been significantly 
revised to include new information on the dominant 
factors controlling leaf reflectance and the introduc- 
tion of numerous new indices with graphic examples. 
Several new texture transforms are introduced (e.g., 
Moran’s J Spatial Autocorrelation) and new informa- 
tion is provided on the extraction of texture from imag- 
es using Grey-level Co-occurrence Matrices (GLCM). 
The chapter concludes with a new discussion on land- 
scape ecology metrics that can be extracted from re- 
motely sensed data. 


Chapter 9: Thematic Information 
Extraction: Pattern Recognition 


Updated information on the American Planning Asso- 
ciation Land-Based Classification Standard (NLCS), 
the U.S. NM: tional Land Cover Di tabase (NL CD Clas- 
sification System, NOAA’s Coastal Change Analysis 
Program (C-CAP) Classification Scheme, and the 
IGBP Land-Cover Classification System is included. 
New methods of feature (band) selection are intro- 
duced (e.g., Correlation Matrix Feature Selection). Ad- 
ditional information is provided on Object-Based 
Image Analysis (OBIJA) classification methods, includ- 
ing new OBIA application examples. 


Chapter 10: Information 
Extraction Using Artificial 
Intelligence 


New information is provided on image classification 
using machine-learning decision trees, regression trees, 
Random Forest (trees), and Support Vector Machines 
(SVM). Detailed information is now provided on a 
number of machine-learning, data-mining decision 
tree/regression tree programs that can be used to devel- 
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op production rules (e.g., CART, S-Plus, R Develop- 
ment Core Team, C4.5, C5.0, Cubist). New 
information about advances in neural network analysis 
of remote sensor data is included for Multi-layer Per- 
ceptrons, Kohonen’s Self-Organizing Map, and fuzzy 
ARTMAP neural networks. A new discussion about 
the advantages and disadvantages of artificial neural 
networks is provided. 


Chapter 11: Information 
Extraction Using Imaging 
Spectroscopy 


Advances in airborne and satellite hyperspectral data 
collection are discussed. Advances in the methods used 
to process and analyze hyperspectral imagery are pro- 
vided, including: end-member selection and analysis, 
mapping algorithms, Spectral Mixture Analysis 
(SMA), continuum removal, spectroscopic library 
matching techniques, machine-learning hyperspectral 
analysis techniques, new hyperspectral indices, and de- 
rivative spectroscopy. 


Chapter 12: Change Detection 


This book has always contained detailed digital change 
detection information. New information is provided on 
the impact of sensor system look angle and amount of 
tree or building obscuration. Advances in binary 
“change/no-change” algorithms are provided including 
new analytical methods used to identify the change 
thresholds and new commercial change detection 
products such as ESRI’s Change Matters and MDA’s 
National Urban Change Indicator. Significant advances 


in thematic “from-to” change detection algorithms are 
discussed including photogrammetric and LiDAR- 
grammetric change detection, OBIA post-classification 
comparison change detection, and Neighborhood 
Correlation Image (NCI) change detection. 


Chapter 13: Remote Sensing- 
Derived Thematic Map Accuracy 
Assessment 


There is a significant amount of literature and debate 
about the best method(s) to use to determine the accu- 
racy of remote sensing-derived thematic map produced 
from a single date of imagery or a thematic map de- 
rived from multiple dates of imagery (i.e., change de- 
tection). The accuracy assessment alternatives and 
characteristics of the debate are discussed more thor- 
oughly. 


Appendix: Sources of Imagery and 
Other Geospatial Information 


Remote sensing data is analyzed best when used in 
conjunction with other geospatial information. To this 
end, a new appendix is provided that contains a list of 
selected geospatial datasets that can be evaluated and/ 
or downloaded via the Internet, including: digital ele- 
vation information, hydrology, land use/land cover and 
biodiversity/habitat, road network and population de- 
mographic data, and several types of publicly- and 
commercially-available remote sensor-data. Map or 
image examples of the datasets are presented where ap- 
propriate. 
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1 REMOTE SENSING AND DIGITAL 
IMAGE PROCESSING 


Scientists observe nature and man-made phenomena, 
make measurements, and then attempt to test hypothe- 
ses concerning these phenomena. The data collection 
may take place directly in the field (referred to as in situ 
or in-place data collection), and/or at some remote dis- 
tance from the subject matter (commonly referred to as 
remote sensing of the environment). Remote sensing 
technology is now used routinely to obtain accurate, 
timely information for a significant variety of applica- 
tions, including: the study of daily weather and long- 
term climate change; urban-suburban land-use/land 
cover monitoring; ecosystem modeling of vegetation, 
water, snow/ice; food security; military reconnaissance; 
and many others (NRC, 2007ab, 2009, 2013, 2014). 
The majority of the remotely sensed data are analyzed 
using digital image processing techniques. 


¥ Overview 


This chapter introduces basic in situ data-collection 
considerations. Remote sensing is then formally de- 
fined along with its advantages and limitations. The re- 
mote sensing—process is introduced with particular 
attention given to: a) the statement of the problem, b) 
identification of in situ and remote sensing data re- 
quirements, c) remote sensing data collection using sat- 
ellite and airborne sensor systems, d) the conversion of 
remote sensing data into information using analog 
and/or digital image processing techniques, and e) ac- 
curacy assessment and information presentation alter- 
natives. Earth observation economics are considered 
along with remote sensing and digital image processing 
careers in the public and private sectors. The organiza- 
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tion of the book is reviewed along with its earth re- 
source analysis perspective. 


i In Situ Data Collection 


One form of in situ data collection involves the scientist 
questioning the phenomena of interest. For example, a 
census enumerator may go door to door, asking people 
questions about their age, sex, education, income, etc. 
These data are recorded and used to document the de- 
mographic characteristics of the population. 


Conversely, a scientist may use a transducer or other in 
situ instrument to make measurements. Transducers 
are usually placed in direct physical contact with the 
object of interest. Many different types of transducers 
are available. For example, a scientist could use a ther- 
mometer to measure the temperature of the air, soil, or 
water; an anemometer to measure wind speed; or a 
psychrometer to measure air humidity. The data re- 
corded by the transducers may be an analog electrical 
signal with voltage variations related to the intensity of 
the property being measured. Often these analog sig- 
nals are transformed into digital values using analog- 
to-digital (A-to-D) conversion procedures. Jn situ data 
collection using transducers relieves the scientist of 
monotonous data collection often in inclement weath- 
er. Also, the scientist can distribute the transducers at 
geographic locations throughout the study area, allow- 
ing the same type of measurement to be obtained at 
many locations at the same time. Sometimes data from 
the transducers are telemetered electronically to a cen- 
tral collection point for rapid evaluation and archiving. 


2 INTRODUCTORY DIGITAL IMAGE PROCESSING 


a. Vegetation height and GPS location b. Spectral reflectance measurement 
measurement near Monticello, UT. of grassland near Monticello, UT. 
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the grassland in Figure 1-1b. used to measure leaf-area-index (LAI) 


near Monticello, UT. 


FIGURE 1-1 In situ (in-place) data are collected in the field. a) Vegetation height is being measured using a simple metal rul- 
er and geographic location is measured using a hand-held GPS unit. b) Vegetation spectral reflectance information is being 
collected using a spectroradiometer held approximately 1 m above the canopy. The in situ spectral reflectance measure- 
ments may be used to calibrate the spectral reflectance measurements obtained from a remote sensing system. c) Spectral 
reflectance characteristics of the area on the ground in Figure 1-1b. d) The leaf-area-index (LAI) of the grassland can be mea- 
sured using a ceptometer that records the number of “sunflecks” that pass through the vegetation canopy. Ceptometer mea- 
surements are made just above the canopy as shown and on the ground below the canopy. These measurements are then 
used to compute in situ LAI that can be used to calibrate remote sensing-derived LAI estimates. 
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Several examples of in situ data collection on a grass- 
land near Monticello, UT, are demonstrated in Figure 
1-1. A stake is used to mark the location of in situ sam- 
ple “UPL 5” in Figure 1-la. A hand-held Global Posi- 
tioning System (GPS) unit is placed at the base of the 
stake to obtain precise x,y, and z location information 
about the sample with a horizontal accuracy of ap- 
proximately +0.25 m. A dimensionally-stable metal 
ruler is used to measure the height of the grass at the 
sample location. A hand-held spectroradiometer is 
used to measure the spectral reflectance characteristics 
of the materials within the Instantaneous-Field-Of- 
View (IFOV) of the radiometer on the ground (Figure 
1-1b). The spectral reflectance characteristics of the 
grassland within the IFOV from 400 — 2,400 nm are 
shown in Figure 1-1lc. A hand-held ceptometer is being 
used to collect information about incident skylight 
above the vegetation canopy in Figure 1-ld. The scien- 
tist will then place the ceptometer on the ground be- 
neath the vegetation canopy present. Approximately 80 
photo-diodes along the linear ceptometer measure the 
amount of light that makes its way through the vegeta- 
tion canopy to the ceptometer. The above- and below- 
canopy ceptometer measurements are used to compute 
the Leaf-Area-Index (LAI) at the sample location. The 
greater the amount of vegetation, the greater the LAI. 
Interestingly, all of these measurement techniques are 
non-destructive (i.e., vegetation clipping or harvesting 
is not required). 


Data collection by scientists in the field or by instru- 
ments placed in the field provides much of the data for 
physical, biological, and social science research. How- 
ever, it is important to remember that no matter how 
careful the scientist is, error may be introduced during 
the in situ data-collection process. First, the scientist in 
the field can be intrusive. This means that unless great 
care is exercised, the scientist can actually change the 
characteristics of the phenomenon being measured 
during the data-collection process. For example, a sci- 
entist collecting a vegetation spectral reflectance read- 
ing could inadvertently step on the sample site, 
disturbing the vegetation prior to data collection. 


Scientists may also collect data in the field using biased 
procedures. This introduces method-produced error. It 
could involve the use of a biased sampling design or 
the systematic, improper use of a piece of equipment. 
Finally, the in situ data-collection measurement device 
may be calibrated incorrectly. This can result in serious 
measurement-device calibration error. 


Intrusive in situ data collection, coupled with human 
method-produced error and measurement-device mis- 
calibration, all contribute to in situ data-collection er- 
ror. Therefore, it is a misnomer to refer to in situ data 
as ground truth data. Instead, we should simply refer 
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to it as in situ ground reference data, acknowledging 
that it contains error. 


Remote Sensing Data 
Collection 


Fortunately, it is possible to collect information about 
an object or geographic area from a distant vantage 
point using remote sensing instruments (Figure 1-2). 
Remote sensing data collection was originally per- 
formed using cameras mounted in suborbital aircraft. 
Photogrammetry was defined in the early editions of 
the Manual of Photogrammetry as: 


the art or science of obtaining reliable measure- 
ment by means of photography (American So- 
ciety of Photogrammetry, 1952; 1966). 


Photographic interpretation is defined as: 


the act of examining photographic images for 
the purpose of identifying objects and judging 
their significance (Colwell, 1960). 


Remote sensing was formally defined by the American 
Society for Photogrammetry and Remote Sensing 
(ASPRS) as: 


the measurement or acquisition of informa- 
tion of some property of an object or phenom- 
enon, by a recording device that is not in 
physical or intimate contact with the object or 
phenomenon under study (Colwell, 1983). 


In 1988, ASPRS adopted a combined definition of pho- 
togrammetry and remote sensing: 


Photogrammetry and remote sensing are the 
art, science, and technology of obtaining reli- 
able information about physical objects and 
the environment, through the process of re- 
cording, measuring and interpreting imagery 
and digital representations of energy patterns 
derived from non-contact sensor systems (Col- 
well, 1997). 


But where did the term remote sensing come from? The 
actual coining of the term goes back to an unpublished 
paper in the early 1960s by the staff of the Office of 
Naval Research (ONR) Geography Branch (Pruitt, 
1979; Fussell et al., 1986). Evelyn L. Pruitt was the au- 
thor of the paper. She was assisted by staff member 
Walter H. Bailey. Aerial photo interpretation had be- 
come very important in World War II. The space age 
was just getting under way with the 1957 launch of 
Sputnik (U.S.S.R.), the 1958 launch of Explorer 1 


4 INTRODUCTORY DIGITAL IMAGE PROCESSING 


(U.S.), and the collection of photography from the then 
secret CORONA program initiated in 1960 (Table 1-1, 
page 11). In addition, the Geography Branch of ONR 
was expanding its research using instruments other 
than cameras (e.g., scanners, radiometers) and into re- 
gions of the electromagnetic spectrum beyond the visi- 
ble and near-infrared regions (e.g., thermal infrared, 
microwave). Thus, in the late 1950s it had become ap- 
parent that the prefix “photo” was being stretched too 
far in view of the fact that the root word, photography, 
literally means “to write with [visible] light” (Colwell, 
1997). Evelyn Pruitt (1979) wrote: 


The whole field was in flux and it was difficult 
for the Geography Program to know which 
way to move. It was finally decided in 1960 to 
take the problem to the Advisory Committee. 
Walter H. Bailey and I pondered a long time 
on how to present the situation and on what to 
call the broader field that we felt should be en- 
compassed in a program to replace the aerial 
photointerpretation project. The term ‘photo- 
graph’ was too limited because it did not cover 
the regions in the electromagnetic spectrum 
beyond the ‘visible’ range, and it was in these 
nonvisible frequencies that the future of inter- 
pretation seemed to lie. ‘Aerial’ was also too 
limited in view of the potential for seeing the 
Earth from space. 


The term remote sensing was promoted in a series of 
symposia sponsored by ONR at the Willow Run Labo- 
ratories of the University of Michigan in conjunction 
with the National Research Council throughout the 
1960s and early 1970s, and has been in use ever since 
(Estes and Jensen, 1998). 


Remote sensing instruments such as cameras, multi- 
spectral and hyperspectral sensors, thermal-infrared 
detectors, Radio Detection and Ranging (RADAR) 
sensors, and Light Detection and Ranging (LiDAR) 
instruments are flown onboard satellites or suborbital 
aircraft such as airplanes, helicopters, and Unmanned 
Aerial Vehicles (UAVs) (Figure 1-2). Sound Navigation 
and Ranging (SONAR) sensors are placed onboard 
ships and submarines to map the bathymetry of sub- 
surface terrain. 


Observations About Remote Sensing 


The following brief discussion focuses on various terms 
found in the formal definitions of remote sensing. 


Remote Sensing: Art and/or Science? 

Science: A science is a broad field of human knowl- 
edge concerned with facts held together by principles 
(rules). Scientists discover and test facts and principles 
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FIGURE 1-2 A remote sensing instrument collects informa- 
tion about an object or phenomenon within the IFOV of the 
sensor system without being in direct physical contact with 

it. The remote sensing instrument may be located just a few 
meters above the ground or onboard an aircraft or satellite 

platform. 
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FIGURE 1-3 a) Interaction model depicting the relationship of the geographic information sciences (geodesy, surveying, car- 
tography, GIS, and remote sensing) as they relate to mathematics and logic and the physical, biological, and social sciences. 

b) Geodesy and surveying provide geodetic control and detailed local geospatial information. c) Cartographers map geospa- 
tial information and Geographic Information Systems are used to model geospatial information. d) Satellite and suborbital re- 
mote sensing systems provide much of the useful geospatial information used by cartographers, GIS practitioners, and other 


scientists. 


by the scientific method, an orderly system of solving 
problems. Scientists generally feel that any subject that 
humans can study by using the scientific method and 
other special rules of thinking may be called a science. 
The sciences include 1) mathematics and logic, 2) physi- 
cal sciences, such as physics and chemistry, 3) biological 
sciences, such as botany and zoology, and 4) social sci- 
ences, such as geography, sociology, and anthropology 
(Figure 1-3a). Interestingly, some persons do not con- 
sider mathematics and logic to be sciences. But the 


fields of knowledge associated with mathematics and 
logic are such valuable tools for science that we cannot 
ignore them. The human race’s earliest questions were 
concerned with “how many” and “what belonged to- 
gether.” They struggled to count, to classify, to think 
systematically, and to describe exactly. In many re- 
spects, the state of development of a science is indicat- 
ed by the use it makes of mathematics. A science seems 
to begin with simple mathematics to measure, then 
works toward more complex mathematics to explain. 
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Remote sensing is a tool or technique similar to mathe- 
matics. Using sophisticated sensors to measure the 
amount of electromagnetic energy exiting an object or 
geographic area from a distance and then extracting 
valuable information from the data using mathemati- 
cally and statistically based algorithms is a scientific 
activity. Remote sensing functions in harmony with 
several other geographic information sciences (often 
referred to as the GISciences), including geodesy, sur- 
veying, cartography, and Geographic Information Sys- 
tems (GIS) (Figure 1-3b-d). The model shown in 
Figure 1-3a suggests there is interaction between the 
mapping sciences, where no subdiscipline dominates 
and all are recognized as having unique yet overlapping 
areas of knowledge and intellectual activity as they are 
used in physical, biological, and social science re- 
search. 


Significant advances will continue to be made in all of 
these technologies. In addition, the technologies will 
become more integrated with one another. For exam- 
ple, GIS network analysis applications have benefited 
tremendously from more accurate road network cen- 
terline data obtained using GPS units mounted on spe- 
cially prepared cars or derived from high spatial 
resolution remote sensor data. Entire industries are 
now dependent on high spatial resolution satellite and 
airborne remote sensor data as geographic background 
imagery for their search engines (e.g., Google Earth, 
Google Maps, Bing Maps). Remote sensing data collec- 
tion has benefited from advancements in GPS mea- 
surements that are used to improve the geometric 
accuracy of images and image-derived products that 
are used so heavily in GIS applications. Terrestrial sur- 
veying has been revolutionized by improvements in 
GPS technology, where measurements as accurate as 
+1 to 3 cm in x, y, and z are now possible. 


The theory of science suggests that scientific disciplines 
go through four classic developmental stages. Wolter 
(1975) suggested that the growth of a scientific disci- 
pline, such as remote sensing, that has its own tech- 
niques, methodologies, and intellectual orientation 
seems to follow the sigmoid or logistic curve illustrated 
in Figure 1-4. The growth stages of a scientific field are: 
Stage 1—a preliminary growth period with small incre- 
ments of literature; Stage 2—a period of exponential 
growth when the number of publications doubles at 
regular intervals; Stage 3—a period when the rate of 
growth begins to decline but annual increments remain 
constant; and Stage 4—a final period when the rate of 
growth approaches zero. The characteristics of a schol- 
arly field during each of the stages may be briefly de- 
scribed as follows: Stage 1—little or no social 
organization; Stage 2—groups of collaborators and ex- 
istence of invisible colleges, often in the form of ad hoc 
institutes, research units, etc.; Stage 3—increasing spe- 
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FIGURE 1-4 Developmental stages of a scientific discipline 
(Source: Wolter, 1975; Jensen and Dahlberg, 1983). 


cialization and increasing controversy; and Stage 4— 
decline in membership in both collaborators and invis- 
ible colleges. 


Using this logic, it appears that remote sensing is in 
Stage 2, experiencing exponential growth since the 
mid-1960s with the number of publications doubling at 
regular intervals. Empirical evidence consists of: 1) the 
organization of many specialized institutes and centers 
of excellence associated with remote sensing, 2) the or- 
ganization of numerous professional societies devoted 
to remote sensing research, 3) the publication of nu- 
merous new scholarly remote sensing journals, 4) sig- 
nificant technological advancement such as improved 
sensor systems and methods of image analysis, and 5) 
robust self-examination. We may be approaching Stage 
3 with increasing specialization and theoretical contro- 
versy. However, the rate of growth of remote sensing 
has not begun to decline. In fact, there has been a tre- 
mendous surge in the numbers of persons specializing 
in remote sensing and commercial firms using remote 
sensing during the last three decades. Significant 1m- 
provements in the spatial resolution of satellite remote 
sensing (e.g., high resolution 1 x 1 m panchromatic da- 
ta) has brought even more social science GIS practitio- 
ners into the fold. Hundreds of new peer-reviewed 
remote sensing research articles are published every 
month. 


Art: The process of visual photo or image interpreta- 
tion brings to bear not only scientific knowledge, but 
all of the background that a person has obtained 
through his or her lifetime. Such learning cannot be 
measured, programmed, or completely understood. 
The synergism of combining scientific knowledge with 
the real-world analyst experience allows the interpreter 
to develop heuristic rules of thumb to extract valuable 
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information from the imagery. It is a fact that some 1m- 
age analysts are superior to other image analysts be- 
cause they: 1) understand the scientific principles 
better, 2) are more widely traveled and have seen many 
landscape objects and geographic areas, and/or 3) can 
synthesize scientific principles and real-world knowl- 
edge to reach logical and correct conclusions. Thus, re- 
mote sensing image interpretation is both an art and a 
science. 


Information About an Object or Area 

Sensors can obtain very specific information about an 
object (e.g., the diameter of an oak tree crown) or the 
geographic extent of a phenomenon (e.g., the polygo- 
nal boundary of an entire oak forest). The electromag- 
netic energy emitted or reflected from an object or 
geographic area is used as a surrogate for the actual 
property under investigation. The electromagnetic en- 
ergy measurements are typically turned into informa- 
tion using visual and/or digital image processing 
techniques. 


The Instrument (Sensor) 

Remote sensing is performed using an instrument, of- 
ten referred to as a sensor. The majority of remote 
sensing instruments record electromagnetic radiation 
(EMR) that travels at a velocity of 3 x 10° ms from 
the source, directly through the vacuum of space or in- 
directly by reflection or reradiation to the sensor. The 
EMR represents a very efficient high-speed communi- 
cations link between the sensor and the remote phe- 
nomenon. In fact, we know of nothing that travels 
faster than the speed of light. Changes in the amount 
and properties of the EMR become, upon detection by 
the sensor, a valuable source of data for interpreting 
important properties of the phenomenon (e.g., temper- 
ature, color). Other types of force fields may be used in 
place of EMR, such as acoustic (sonar) waves. Howevy- 
er, the majority of remotely sensed data collected for 
Earth resource applications is the result of sensors that 
record electromagnetic energy. 


Distance: How Far Is Remote? 

Remote sensing occurs at a distance from the object or 
area of interest. Interestingly, there is no clear distinc- 
tion about how great this distance should be. The inter- 
vening distance could be 1 cm, 1 m, 100 m, or more 
than | million m from the object or area of interest. 
Much of astronomy is based on remote sensing. In 
fact, many of the most innovative remote sensing sys- 
tems and visual and digital image processing methods 
were originally developed for remote sensing extrater- 
restrial landscapes such as the moon, Mars, Saturn, Ju- 
piter, etc. (especially by NASA’s Jet Propulsion 
Laboratory personnel). This text, however, is con- 
cerned primarily with remote sensing of the terrestrial 
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Earth, using sensors that are placed on suborbital air- 
breathing aircraft or orbital satellite platforms placed 
in the vacuum of space. 


Remote sensing and digital image processing tech- 
niques can also be used to analyze inner space. For ex- 
ample, an electron microscope can be used to obtain 
photographs of extremely small objects on the skin, in 
the eye, etc. An x-ray instrument is a remote sensing 
system where the skin and muscle are like the atmo- 
sphere that must be penetrated, and the interior bone 
or other matter is the object of interest. 


Remote Sensing Advantages and 
Limitations 


Remote sensing has several unique advantages as well 
as some limitations. 


Advantages 

Remote sensing is unobtrusive if the sensor is passively 
recording the electromagnetic energy reflected from or 
emitted by the phenomenon of interest. This is a very 
important consideration, as passive remote sensing 
does not disturb the object or area of interest. 


Remote sensing devices are programmed to collect 
data systematically, such as within a single 9 x 9 in. 
frame of vertical aerial photography or a matrix (ras- 
ter) of Landsat 5 Thematic Mapper data. This system- 
atic data collection can remove the sampling bias 
introduced in some in situ investigations. 


Remote sensing science is also different from cartogra- 
phy or GIS because these sciences rely on data ob- 
tained or synthesized by others. Remote sensing 
science can provide fundamental, new scientific data or 
information. Under controlled conditions, remote 
sensing can provide fundamental biophysical informa- 
tion, including: x, y location, z elevation or depth, bio- 
mass, temperature, moisture content, etc. In this sense, 
remote sensing science is much like surveying, provid- 
ing fundamental information that other sciences can 
use when conducting scientific investigations. However, 
unlike much of surveying, the remotely sensed data can 
be obtained systematically over very large geographic 
areas rather than just single-point observations. In 
fact, remote sensing—derived information is now criti- 
cal to the successful modeling of numerous natural 
(e.g., water-supply estimation; eutrophication studies; 
nonpoint source pollution) and cultural (e.g., land-use 
conversion at the urban fringe; water-demand estima- 
tion; population estimation; food security) processes 
(NRC, 2007a; 2009). A good example is the digital ele- 
vation model that is so important in many spatially- 
distributed GIS models. Digital elevation models are 
now produced mainly from light detection and ranging 
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(LiDAR) (e.g., Renslow, 2012), stereoscopic aerial 
photography, RADAR measurements, or Interfero- 
metric Synthetic Aperture Radar (IFSAR) imagery. 


Limitations 

Remote sensing science has limitations. Perhaps the 
greatest limitation is that it is often oversold. Remote 
sensing is not a panacea that will provide all the infor- 
mation needed to conduct physical, biological, or so- 
cial science research. It simply provides some spatial, 
spectral, and temporal information of value in a man- 
ner that is hopefully efficient and economical. 


Human beings select the most appropriate remote 
sensing system to collect the data, specify the various 
resolution(s) of the remote sensor data, calibrate the 
sensor, select the satellite or suborbital platform that 
will carry the sensor, determine when the data will be 
collected, and specify how the data are processed. 
Therefore, human method-produced error may be in- 
troduced as the remote sensing instrument and mission 
parameters are specified. 


Powerful active remote sensor systems that emit their 
own electromagnetic radiation (e.g., LIDAR, RADAR, 
SONAR) can be intrusive and affect the phenomenon 
being investigated. Additional research is required to 
determine how intrusive these active sensors can be. 


Remote sensing instruments may become uncalibrated, 
resulting in uncalibrated remote sensor data. Finally, 
remote sensor data may be relatively expensive to col- 
lect and analyze. Hopefully, the information extracted 
from the remote sensor data justifies the expense. The 
greatest expense in a typical remote sensing investiga- 
tion is for well-trained image analysts, not remote sen- 
sor data. 


¥ 


Scientists have been developing procedures for collect- 
ing and analyzing remotely sensed data for more than 
150 years. The first photograph from an aerial plat- 
form (a tethered balloon) was obtained in 1858 by the 
Frenchman Gaspard Felix Tournachon (who called 
himself Nadar). Significant strides in aerial photogra- 
phy and other remote sensing data collection took 
place during World Wars I and I, the Korean Conflict, 
the Cuban Missile Crisis, the Vietnam War, the Gulf 
War, the war in Bosnia, and the war on terrorism. Basi- 
cally, military contracts to commercial companies re- 
sulted in the development of sophisticated electro- 
optical multispectral remote sensing systems and ther- 
mal infrared and microwave (radar) sensor systems 


The Remote Sensing 
Process 


whose characteristics are summarized in Chapter 2. 
While the majority of the remote sensing systems may 
have been initially developed for military reconnais- 
sance applications, the systems are now also heavily 
used for monitoring the Earth’s natural resources. 


The remote sensing data-collection and analysis proce- 
dures used for Earth resource applications are often 
implemented in a systematic fashion that can be 
termed the remote sensing process. The procedures in 
the process are summarized here and in Figure 1-5: 


¢ The hypothesis to be tested is defined using a spe- 
cific type of logic (e.g., inductive, deductive) and an 
appropriate processing model (e.g., deterministic, 
stochastic). 


¢ Jn situ and collateral information necessary to cali- 
brate the remote sensor data and/or judge its geo- 
metric, radiometric, and thematic characteristics are 
collected. 


¢ Remote sensor data are collected passively (e.g., dig- 
ital cameras) or actively (eg., RADAR, LiDAR) 
using analog or digital remote sensing instruments, 
ideally at the same time as the in situ data. 


¢ In situ and remotely sensed data are processed using 
a variety of techniques, including: a) analog image 
processing, b) digital image processing, c) modeling, 
and d) n-dimensional visualization. 

¢ Metadata, processing lineage, and the accuracy of 
the information are provided and the results com- 
municated using images, graphs, statistical tables, 
GIS databases, Spatial Decision Support Systems 
(SDSS), ete. 


It is useful to review the characteristics of these remote 
sensing process procedures. 


Statement of the Problem 


Sometimes the general public and even children look at 
aerial photography or other remote sensor data and ex- 
tract useful information. They do this without a formal 
hypothesis in mind. Often, however, they interpret the 
imagery incorrectly because they do not understand 
the nature of the remote sensing system used to collect 
the data or appreciate the vertical or oblique perspec- 
tive of the terrain recorded in the imagery. 


Scientists who use remote sensing, on the other hand, 
are usually trained in the scientific method—a way of 
thinking about problems and solving them. They use a 
formal plan that typically has at least five elements: 1) 
stating the problem, 2) forming the research hypothesis 
(i.e., a possible explanation), 3) observing and experi- 
menting, 4) interpreting data, and 5) drawing conclu- 
sions. It is not necessary to follow this formal plan 
exactly. 
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The Remote Sensing Process 
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- Expert systems - Thematic maps 
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- Machine learning - Animations 
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- Hyperspectral analysis 
- Change detection 


- Modeling ° Statistics 
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¢ Hypothesis Testing 
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FIGURE 1-5 Scientists generally use the remote sensing process to extract information from remotely-sensed images. 


The scientific method is normally used in conjunction 
with environmental models that are based on two pri- 
mary types of logic: 


¢ deductive logic, and 
* inductive logic. 


Models based on deductive and/or inductive logic can 
be further subdivided according to whether they are 
processed deterministically or stochastically. Some sci- 
entists extract new thematic information directly from 
remotely sensed imagery without ever explicitly using 
inductive or deductive logic. They are just interested in 
extracting information from the imagery using appro- 
priate methods and technology. This technological ap- 
proach is not as rigorous, but it is common in applied 
remote sensing. The approach can also generate new 
knowledge. 


Remote sensing is used in both scientific (inductive and 
deductive) and technological approaches to obtain 
knowledge. There is discussion as to how the different 


types of logic used in the remote sensing process yield 
new scientific knowledge (e.g., Fussell et al., 1986; Cur- 
ran, 1987; Fisher and Lindenberg, 1989; Dobson, 
1993; Skidmore, 2002; Wulder and Coops, 2014). 


Identification of In situ and Remote 
Sensing Data Requirements 


If a hypothesis is formulated using inductive and/or de- 
ductive logic, a list of variables or observations are 
identified that will be used during the investigation. Jn 
situ observation and/or remote sensing may be used to 
collect information on the most important variables. 


Scientists using remote sensing technology should be 
well trained in field and laboratory data-collection pro- 
cedures. For example, if a scientist wants to map the 
surface temperature of a lake, it is usually necessary to 
collect accurate empirical in situ lake-temperature mea- 
surements at the same time the remote sensor data are 
collected. The in situ data may be used to: 1) calibrate 
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the remote sensor data, and/or 2) perform an unbiased 
accuracy assessment of the final results (Congalton 
and Green, 2009). Remote sensing textbooks provide 
some information on field and laboratory sampling 
techniques (e.g., Jensen, 2007). The in situ sampling 
procedures, however, are learned best through formal 
courses in the sciences (e.g., chemistry, biology, forest- 
ry, soils, hydrology, meteorology). It is also important 
to know how to collect accurate socioeconomic and 
demographic information in urban environments 
based on training in human geography, sociology, etc. 
(e.g., McCoy, 2005; Azar et al., 2013). 


Most in situ data are now collected in conjunction with 
global positioning system (GPS) x, y, z data (Jensen 
and Jensen, 2013). Scientists should know how to col- 
lect the GPS data at each in situ data-collection station 
and turn it into useful geospatial information. 


Collateral Data Requirements 

Many times collateral data (often called ancillary da- 
ta), such as digital elevation models, soil maps, geology 
maps, political boundary files, and block population 
statistics, are of value in the remote sensing process. 
Ideally, the geospatial collateral data reside in a GIS 
(Jensen and Jensen, 2013). 


Remote Sensing Data Requirements 

Once we have a list of variables, it is useful to deter- 
mine which of them can be remotely sensed. Remote 
sensing can provide information on two different class- 
es of variables: biophysical and hybrid. 


Biophysical Variables: Certain biophysical variables 
can be measured directly by a remote sensing system. 
This means that the remotely sensed data can provide 
fundamental biological and/or physical (biophysical) 
information directly, without having to use other surro- 
gate or ancillary data (Wulder and Coops, 2014). For 
example, a thermal infrared remote sensing system can 
record the apparent surface temperature of a rock out- 
crop or agricultural field by measuring the radiant en- 
ergy exiting its surface. Similarly, it is possible to 
conduct remote sensing in a very specific region of the 
spectrum and identify the amount of water vapor in 
the atmosphere. It is also possible to measure soil 
moisture content directly using active and passive mi- 
crowave remote sensing techniques. NASA’s Moderate 
Resolution Imaging Spectrometer (MODIS) can be 
used to model Absorbed Photosynthetically Active 
Radiation (APAR) and LAI. The precise x, y location, 
and height or elevation (z) of an object can be extract- 
ed directly from stereoscopic aerial photography, over- 
lapping satellite imagery (e.g., SPOT), LiDAR data, or 
IFSAR imagery. 


Table 1-1 is a list of selected biophysical variables that 
can be remotely sensed and useful sensors to acquire 
the data. Characteristics of many of these remote sens- 
ing systems are discussed in Chapter 2. Great strides 
have been made in remotely sensing many of these bio- 
physical variables. They are important to the national 
and international effort under way to model the global 
environment (e.g., NRC, 2012; Brewin et al., 2013). 


Hybrid Variables: The second general group of vari- 
ables that can be remotely sensed includes hybrid vari- 
ables, created by systematically analyzing more than 
one biophysical variable. For example, by remotely 
sensing a plant’s chlorophyll absorption characteristics, 
temperature, and moisture content, it might be possi- 
ble to model these data to detect vegetation stress, a 
hybrid variable. The variety of hybrid variables is large; 
consequently, no attempt is made to identify them. It is 
important to point out, however, that nominal-scale 
land use and land cover are hybrid variables. For exam- 
ple, the land cover of a particular area on an image 
may be derived by evaluating several of the fundamen- 
tal biophysical variables at one time (e.g., object loca- 
tion [x, y], height [z], tone and/or color, biomass, and 
perhaps temperature). So much attention has been 
placed on remotely sensing this hybrid nominal-scale 
variable that the interval- or ratio-scaled biophysical 
variables were largely neglected until the mid-1980s. 
Nominal-scale land-use and land-cover mapping are 
important capabilities of remote sensing technology 
and should not be minimized. Many social and physi- 
cal scientists routinely use such data in their research. 
However, there is now a dramatic increase in the ex- 
traction of interval- and ratio-scaled biophysical data 
that are incorporated into quantitative models that can 
accept spatially distributed information. 


Remote Sensing Data Collection 


Remotely sensed data are collected using passive or ac- 
tive remote sensing systems. Passive sensors record 
electromagnetic radiation that is reflected or emitted 
from the terrain. For example, cameras and video re- 
corders can be used to record visible and near-infrared 
energy reflected from the terrain. A multispectral scan- 
ner can be used to record the amount of thermal radi- 
ant flux exiting the terrain. Active sensors, such as 
microwave (RADAR), LiDAR, or SONAR, bathe the 
terrain in machine-made electromagnetic energy and 
then record the time-lapsed amount of radiant flux 
scattered back toward the sensor system. 


Remote sensing systems collect analog (e.g., hard-copy 
aerial photography or video data) and/or digital data 
(e.g., a matrix [raster] of brightness values obtained us- 
ing a scanner, linear array, or area array). 
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TABLE 1-1 Selected biophysical and hybrid variables and potential remote sensing systems used to obtain the information. 


Biophysical Variables 


Potential Remote Sensing Systems 


x, y, z Location and Geodetic Control 
x, y Location from Orthocorrected Imagery 


- Global Positioning Systems (GPS) 

- Analog and digital stereoscopic aerial photography, GeoEye-1, WorldView-2, 
SPOT 6 and 7, Landsat 7 and 8, ResourceSat, ERS-1 and -2, Moderate Resolution 
Imaging Spectrometer (MODIS), Pleiades, LIDAR, RADARSAT-1 and -2 


z Topography/Bathymetry 
- Digital Elevation Model (DEM) 


- Digital Bathymetric Model (DBM) 


- GPS, stereoscopic aerial photography, LIDAR, SPOT 6, RADARSAT, GeoEye-1, 
WorldView-2 and -3, Shuttle Radar Topography Mission (SRTM), IFSAR 
- SONAR, bathymetric LIDAR, stereoscopic aerial photography 


Vegetation 
- Pigments (e.g., chlorophyll a and b) 


- Canopy structure and height 

- Biomass derived from vegetation indices 
-LAl 

- Absorbed photosynthetically active radiation 
- Evapotranspiration 


- Color aerial photography, Landsat 8, GeoEye-1, WorldView-2 and -3, Advanced 
Spaceborne Thermal Emission and Reflection Radiometer (ASTER), MODIS, Pleia- 
des, airborne hyperspectral systems (e.g., AVIRIS, HyMap, CASI) 

- Stereoscopic aerial photography, LIDAR, RADARSAT, IFSAR 

- Color-infrared (CIR) aerial photography, Landsat 8, GeoEye-1, WorldView-2, 
Advanced Very High Resolution Radiometer (AVHRR), Multiangle Imaging 
Spectroradiometer (MISR), Pleiades, satellite (EO-1 Hyperion) and airborne hyper- 
spectral systems (e.g., AVIRIS, HyMap, CASI-1500) 


Surface Temperature (land, water, atmosphere) 


- ASTER, AVHRR, GOES, Hyperion, Landsat 8, MODIS, airborne thermal infrared 


Soil and Rocks 
- Moisture 


- Mineral composition 
- Taxonomy 
- Hydrothermal alteration 


- pele pescve microwave (SSM/1), RADARSAT-2, MISR, ALMAZ, Landsat 8, 
ERS-1 and -2 

- ASTER, MODIS, airborne and satellite hyperspectral systems 

- Color and CIR aerial photography, airborne piperse etal systems 

- Landsat 8, ASTER, MODIS, airborne and satellite hyperspectral systems 


Surface Roughness 


- Aerial photography, RADARSAT-1 and -2, IKONOS-2, WorldView-2, ASTER 


Atmosphere 

- Aerosols (e.g., optical depth) 

- Clouds (e.g., fraction, optical thickness) 
- Precipitation 

- Water vapor (precipitable water) 


- NPP, MISR, GOES, AVHRR, MODIS, CERES, MOPITT, MERIS 

- NPP, GOES, AVHRR, MODIS, MISR, CERES, MOPITT, UARS, MERIS 

- Tropical Rainfall Measurement Mission (TRMM), GOES, AVHRR, SSM/1, MERIS 
- NPP, GOES, MODIS, MERIS 


- Ozone - NPP, MODIS 
Water 
- Color - Color and CIR aerial photography, Landsat 8, SPOT 6, GeoEye-1, WorldView-2, 


- Surface hydrology 

- Suspended minerals 

- Chlorophyll/gelbstoffe 

- Dissolved organic matter 


Pleiades, ASTER, MODIS, airborne and satellite hyperspectral systems, AVHRR, 
NPP, GOES, bathymetric LIDAR, MISR, CERES, TOPEX/POSEIDON, MERIS 


Snow and Sea Ice 
- Extent and characteristics 


- Color and CIR aerial photography, NPP, AVHRR, GOES, Landsat 8, SPOT 6, Geo- 
es WorldView-2, Pleiades, ASTER, MODIS, MERIS, ERS-1 and -2, RADARSAT-1 
and -2 


Volcanic Effects 
- Temperature, gases 


- ASTER, Landsat 8, MISR, Hyperion, MODIS, thermal hyperspectral systems 


BRDF (bidirectional reflectance distribution function) 


- MISR, MODIS, CERES 


Selected Hybrid Variables 


Potential Remote Sensing Systems 


Land Use 
- Commercial, residential, transportation, etc. 
- Cadastral (property) 


Land Cover 
- Agriculture, forest, urban, etc. 


- High spatial resolution panchromatic, color and /or CIR stereoscopic aerial pho- 
tography, high spatial resolution satellite imagery (<1 x 1 m: GeoEye-1, World- 
View-3), SPOT 6 and 7, LiDAR, high spatial resolution hyperspectral systems 


- Color and CIR aerial photography, Landsat 8, SPOT 6 and 7, ASTER, AVHRR, 
RADARSAT, GeoEye-1, WorldView-2 and -3, Pleiades, LIDAR, IFSAR, MODIS, 
MISR, MERIS, airborne and satellite hyperspectral systems 


Vegetation 
=rSthess 
- composition/type 


- Color and CIR photography, Landsat 8, GeoEye-1, WorldView-2, AVHRR, Sea- 
WiFS, MISR, MODIS, ASTER, MERIS, airborne and satellite hyperspectral systems 


12 INTRODUCTORY DIGITAL IMAGE PROCESSING 


The amount of electromagnetic radiance, L (watts m™ 
sr!; watts per meter squared per steradian), recorded 
within the IFOV of an optical remote sensing system 
(e.g., a pixel in a digital image), is a function of: 

L= fas 


t, B, 0, P, Q) (1.1) 


X,Y, 2? 


where 


X = wavelength (spectral response measured in various 
bands or at specific frequencies). Wavelength (1) and 
frequency (v ) may be used interchangeably based on 
their relationship with the speed of light (c) where 
c=Kxv. 


Sx yz =X y, Z location of the pixel and its size (x, y); 


t = temporal information, i.e., when, how long, and 
how often the data were acquired; 


8 = IFOV: 


0 = set of angles that describe the geometric relation- 
ships between the radiation source (e.g., the Sun), the 
terrain target of interest (e.g., a wheat field), and the re- 
mote sensing system; 


P = polarization of back-scattered energy recorded by 
the sensor; and 


Q = radiometric resolution (precision) at which the 
data (e.g., reflected, emitted, or back-scattered radia- 
tion) are recorded by the remote sensing system. 


It is useful to review characteristics of the parameters 
associated with Equation 1.1 and how they influence 
the nature of the remote sensing data collected. 


Spectral Information and Resolution 

Most remote sensing investigations are based on devel- 
oping a deterministic relationship (i.e., a model) be- 
tween the amount of electromagnetic energy reflected, 
emitted, or back-scattered in specific bands or frequen- 
cies and the chemical, biological, and physical charac- 
teristics of the phenomena under investigation (e.g., a 
wheat field canopy). Spectral resolution is the number 
and dimension (size) of wavelength intervals (referred 
to as bands or channels) in the electromagnetic spec- 
trum to which a remote sensing instrument is sensitive. 


Multispectral remote sensing systems record energy in 
multiple bands of the electromagnetic spectrum. For 
example, in the 1970s and early 1980s, the Landsat 
Multispectral Scanner (MSS) recorded remotely 
sensed data of much of the Earth that is still of signifi- 
cant value for change detection studies. The band- 
widths of the four MSS-bands are displayed in Figure 


1-6a (band 1 = 500 — 600 nm; band 2 = 600 — 700 nm; 
band 3 = 700 — 800 nm; and band 4 = 800 — 1,100 nm). 
The nominal size of a band may be large (i.e., coarse), 
as with the Landsat MSS near-infrared band 4 (800 — 
1,100 nm) or relatively smaller (i.c., finer), as with the 
Landsat MSS band 3 (700 — 800 nm). 


The four multispectral bandwidths associated with a 
typical digital frame camera are also shown in Figure 
1-6a. The camera’s detectors record information in 
four regions of the spectrum (band | = 450 — 515 nm; 
band 2 = 525 — 605 nm; band 3 = 640 — 690 nm; and 
band 4 = 750 — 900 nm). Note that there are gaps be- 
tween the spectral sensitivities of the detectors. Individ- 
ual band images are shown in Figure 1-6c. Natural and 
color-infrared color composites created using the indi- 
vidual bands are shown in Figure 1-6d-e. 


In reality, the spectral sensitivity of an individual band 
does not cut off as neatly as shown in Figure 1-6a. In- 
stead, detectors are sensitive over a region in the spec- 
trum with a maximum intensity at approximately the 
middle of the bandwidth. Therefore, the values used to 
describe the bandwidth of an individual band are usu- 
ally derived using the logic shown in Figure 1-6b where 
the intensity and shape of the Gaussian curve is inves- 
tigated and the Full Width at Half Maximum (FWHM) 
is determined. In this hypothetical example, the 
FWHM of band 2 (green) is from 525 to 605 nm. 
Therefore, we can reasonably say that this is the spec- 
tral bandwidth of this particular band, even though we 
know it has some sensitivity outside of this bandwidth. 
The band center is at 565 nm. Sometimes scientists use 
band center information in their investigations, espe- 
cially when analyzing hyperspectral imagery. 


A hyperspectral remote sensing instrument typically ac- 
quires data in hundreds of spectral bands. For exam- 
ple, a hyperspectral image of Sullivan’s Island, SC, 
collected by NASA’s Airborne Visible and Infrared Im- 
aging Spectrometer (AVIRIS) is shown in Figure 1-7a. 
AVIRIS collects data in 224 bands in the region from 
400 — 2,500 nm spaced just 10 nm apart based on the 
FWHM criteria (NASA AVIRIS, 2014). The band- 
widths of the nine Landsat 8 sensor system are shown 
in Figure 1-7b for comparison (Irons et al., 2012). 
Note how AVIRIS obtains hundreds of measurements 
per pixel over the 400 — 2,500 nm region. Some scien- 
tists like to acquire the complete spectrum of informa- 
tion from 400 — 2,500 nm. Others prefer to work on a 
relatively small subset of carefully placed bands such 
as those associated with Landsat 8 to extract the de- 
sired information. Ultraspectral remote sensing in- 
volves data collection in many hundreds of bands. 


Certain regions or spectral bands of the electromagnet- 
ic spectrum are optimal for obtaining information on 
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FIGURE 1-6 a) The relatively coarse spectral bandwidths of the four Landsat Multispectral Scanner (MSS) bands compared 
with the bandwidths of a typical digital frame camera. b) The true spectral bandwidth is the width of the Gaussian-shaped 
spectral profile based on Full Width at Half Maximum (FWHM) intensity. For example, at FWHM the green band has a spectral 
range of 80 nm between 525 and 605 nm. c) Examples of 1 x 1 ft. spatial resolution individual digital frame camera images. d) 
Natural color composite created using the blue, green, and red bands. e) Color-infrared color-composite created using the 


green, red, and near-infrared bands. 


biophysical parameters. The bands are normally select- 
ed to maximize the contrast between the object of in- 
terest and its background (i.e., object-to-background 
contrast). Careful selection of the spectral bands might 
improve the probability that the desired information 
will be extracted from the remote sensor data. 


Spatial Information and Resolution 

Most remote sensing studies record the spatial attri- 
butes of objects on the terrain. For example, each silver 
halide crystal in an analog aerial photograph and each 
picture element in a digital remote sensor image is lo- 
cated at a specific location in the image and associated 
with specific x, y coordinates on the ground. Once geo- 
graphically referenced to a standard map projection 
(Chapter 7), the spatial information associated with 
each silver halide crystal or pixel is of significant value 


because it allows the remote sensing—derived informa- 
tion to be used with other spatial data in a GIS or spa- 
tial decision support system (Jensen et al., 2002). 


There is a general relationship between the size of an 
object or area to be identified and the spatial resolu- 
tion of the remote sensing system. Spatial resolution is 
a measure of the smallest angular or linear separation 
between two objects that can be resolved by the remote 
sensing system. The spatial resolution of aerial photog- 
raphy may be measured by 1) placing calibrated, paral- 
lel black and white lines on tarps that are placed in the 
field, 2) obtaining aerial photography of the study ar- 
ea, and 3) computing the number of resolvable line 
pairs per millimeter in the photography. It is also possi- 
ble to determine the spatial resolution of imagery by 
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Airborne Visible Infrared Imaging Spectrometer (AVIRIS) Datacube 
of Sullivan’s Island, SC 
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b. Comparison of 224 AVIRIS spectral bands at 10 nm sampling with the sensitivity 
of the nine Landsat 8 non-contiguous bands in the region from 400 to 2,500 nm. 


FIGURE 1-7 a) An AVIRIS hyperspectral datacube of Sullivan's Island, SC. The image on top is a color composite of just three 
of the 224 available bands (RGB = near-infrared, red, green). b) A comparison of the sensitivity of the 224 AVIRIS bands with 
the location of the nine Landsat 8 non-contiguous bands in the region from 400 to 2,500 nm. Source of data: NASA. 


computing its modulation transfer function, which is 
beyond the scope of this text. 


Many satellite remote sensing systems use optics that 
have a constant instantaneous-field-of-view (IFOV) 
(Figure 1-2). Therefore, a sensor system’s nominal spa- 
tial resolution is defined as the dimension in meters (or 
feet) of the ground-projected IFOV where the diameter 
of the circle (D) on the ground is a function of the in- 


stantaneous-field-of-view (B ) times the altitude (H) of 
the sensor above ground level (AGL) (Figure 1-2): 


D=£xH. (1.2) 


Pixels are normally represented on computer screens 
and in hard-copy images as rectangles with length and 
width. Therefore, we typically describe a sensor sys- 
tem’s nominal spatial resolution as being 10 x 10 m or 
30 x 30 m. For example, DigitalGlobe’s World View-2 
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g. 40 x 40 m. 
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Imagery of Harbor Town in Hilton Head, SC, at Various Spatial Resolutions 


f. 20 x 20 m. 


Nominal Spatial Resolution 
(enlarged view) 


80m - 
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instantaneous- 
field-of-view 


80m 


FIGURE 1-8 The original Harbor Town, SC, image was collected at a nominal spatial resolution of 0.3 x 0.3 m (approximately 
1 x 1 ft.) using a digital frame camera. The original imagery was resampled to derive the imagery with the simulated spatial 


resolutions shown. 


has a nominal spatial resolution of 0.46 x 0.46 cm for 
its panchromatic band and 1.85 Xx 1.85 m for the four 
multispectral bands. The Landsat 7 Enhanced Themat- 
ic Mapper Plus (ETM+) has a nominal spatial resolu- 
tion of 15 x 15 m for its panchromatic band and 30 x 
30 m for six of its multispectral bands. Generally, the 
smaller the nominal spatial resolution, the greater the 
spatial resolving power of the remote sensing system. 


Figure 1-8 depicts digital camera imagery of an area in 
Hilton Head, SC, at resolutions ranging from 0.5 x 0.5 


m to 80 x 80 m. Note that there does not appear to be 
a significant difference in the interpretability of 0.5 x 
0.5 m data and | <x 1 m data. However, the urban spa- 
tial information content decreases rapidly when using 5 
x 5m imagery and is practically useless for urban anal- 
ysis at spatial resolutions greater than 10 x 10 m. This 
is the reason historical Landsat MSS data (79 x 79 m) 
are of little value for most urban applications (Jensen 
and Cowen, 1999). 
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Temporal Resolution 
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FIGURE 1-9 The temporal resolution of a remote sensing 
system refers to how often it records imagery of a particular 
area. This example depicts the systematic collection of re- 
mote sensor data every 16 days, presumably at approximate- 
ly the same time of day. NASA Landsat Thematic Mappers 4 
and 5 had 16-day revisit cycles. The NOAA Geostationary 
Operational Environmental Satellites (GOES) collect new im- 
ages every half-hour which are especially useful for monitor- 
ing storm events in near real-time. 


A useful heuristic rule of thumb is that in order to de- 
tect a feature, the nominal spatial resolution of the re- 
mote sensing system should be less than one-half the 
size of the feature measured in its smallest dimension. 
For example, if we want to identify the location of all 
oak trees in a park, the minimum acceptable spatial 
resolution would be approximately one-half the diame- 
ter of the smallest oak tree’s crown. Even this spatial 
resolution, however, will not guarantee success if there 
is no difference between the spectral response of the 
oak tree (the object) and the soil or grass surrounding 
it (1.e., its background). 


Some sensor systems, such as LiDAR, do not com- 
pletely “map” the terrain surface. Rather, the surface is 
“sampled” using laser pulses sent from the aircraft at 
some nominal time interval (Renslow, 2012). The 
ground-projected laser pulse may be very small (e.g., 
10-15 cm in diameter) with samples located approxi- 
mately every 1-6 m on the ground. Spatial resolution 
would appropriately describe the ground-projected la- 
ser pulse (e.g., 15 cm) but sampling density (i.e., num- 
ber of points per unit area) describes the frequency of 
ground observations (Hodgson et al., 2005). 


Because we have spatial information about the location 
of each pixel (x, y) in the image matrix, it is also possi- 
ble to examine the spatial relationship between a pixel 
and its neighbors. Therefore, the amount of spatial au- 
tocorrelation and other spatial geostatistical measure- 
ments can be determined based on the spatial 
information inherent in the imagery. 


Temporal Information and Resolution 

One of the valuable things about remote sensing sci- 
ence is that it obtains a record of Earth landscapes at a 
unique moment in time. Multiple records of the same 
landscape obtained through time can be used to identi- 
fy processes at work and to make predictions. 


The temporal resolution of a remote sensing system 
generally refers to how often and when the sensor re- 
cords imagery of a particular area. The temporal reso- 
lution of the sensor system shown in Figure 1-9 is every 
16 days. Ideally, the sensor obtains data repetitively to 
capture unique discriminating characteristics of the 
object under investigation. For example, agricultural 
crops have unique developmental phenological cycles in 
each geographic region. To measure specific agricultur- 
al variables, it is necessary to acquire remotely sensed 
data at critical dates in the phenological cycle. Analysis 
of multiple-date imagery provides information on how 
the variables are changing through time. Change infor- 
mation provides insight into processes influencing the 
development of the crop. Fortunately, several satellite 
sensor systems such as SPOT 4 and 5, GeoEye-1, Im- 
ageSat and WorldView-2 are pointable, meaning that 
they can acquire imagery off-nadir. Nadir is the point 
directly below the spacecraft. This dramatically in- 
creases the probability that imagery will be obtained 
during a growing season or during an emergency. How- 
ever, off-nadir oblique viewing also introduces bidirec- 
tional reflectance distribution function (BRDF) issues, 
discussed in the next section. 


There are often trade-offs associated with the various 
resolutions that must be made when collecting remote 
sensing data (Figure 1-10). Generally, the higher the 
temporal resolution requirement (e.g., monitoring hur- 
ricanes every half-hour), the lower the spatial resolu- 
tion requirement (e.g., the NOAA GOES weather 
satellite records images with 4 x 4 to 8 x 8 km pixels). 
Conversely, the higher the spatial resolution require- 
ment (e.g., monitoring urban land-use with | x 1 m da- 
ta), the lower the temporal resolution requirement 
(e.g., every 1 to 10 years). For example, Figure 1-11 
documents single-family residential land-use develop- 
ment for an area near White Rock, SC, using high spa- 
tial resolution (1 x 1 ft.) digital aerial photography 
collected in 2004, 2007, and 2009. Some applications 
such as crop type or yield estimation might require rel- 
atively high temporal resolution data (e.g., multiple im- 
ages obtained during a growing season) and moderate 
spatial resolution data (e.g., 250 x 250 m pixels provid- 
ed by NASA’s MODIS sensor). Emergency response 
applications may require very high spatial and tempo- 
ral resolution data collection that generates tremen- 
dous amounts of data (e.g., every 5 hours at 0.5 x 0.5 
m spatial resolution). 
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Spatial and Temporal Resolution for Selected Applications 
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FIGURE 1-10 Spatial and temporal resolution trade-offs must be made when collecting remote sensor data for selected ap- 
plications. For example, applications such as land use mapping generally require high spatial resolution imagery (e.g., 1 to 5 
m) at relatively low temporal resolution (e.g., 1-10 years). Conversely, for weather prediction we are generally content with 
lower spatial resolution imagery (e.g., 5 x 5 km) if it can be collected frequently (e.g., every half-hour). 


Another aspect of temporal information is how many 
observations are recorded from a single pulse of energy 
that is directed at the Earth by an active sensor such as 
LiDAR. For example, most LIDAR sensors emit one 
pulse of laser energy and record multiple responses 
from this pulse. Measuring the time differences be- 
tween multiple responses allows for the determination 
of object heights and terrain structure. 


Radiometric Information and Resolution 

Some remote sensing systems record the reflected, 
emitted, or back-scattered electromagnetic radiation 
with more precision than other sensing systems. This is 
analogous to making a measurement with a ruler. If 
you want precisely to measure the length of an object, 
would you rather use a ruler with 16 or 1,024 subdivi- 
sions on it? 


Radiometric resolution is defined as the sensitivity of a 
remote sensing detector to differences in signal 
strength as it records the radiant flux reflected, emit- 
ted, or back-scattered from the terrain. It defines the 
number of just discriminable signal levels. Therefore, 


radiometric resolution can have a significant impact on 
our ability to measure the properties of scene objects. 
The Landsat 1 Multispectral Scanner launched in 1972 
recorded reflected energy with a precision of 6-bits 
(values ranging from 0 to 63). Landsat 4 and 5 The- 
matic Mapper sensors launched in 1982 and 1984, re- 
spectively, recorded data in 8-bits (values from 0 to 
255) (Figure 1-12). Thus, the Landsat TM sensors had 
improved radiometric resolution (sensitivity) when 
compared with the original Landsat MSS. GeoEye-1 
and World View-2 sensors record information in 11-bits 
(values from 0 to 2,047). Several new sensor systems 
have 12-bit radiometric resolution (values ranging 
from 0 to 4,095). Radiometric resolution is sometimes 
referred to as the level of quantization. High radiomet- 
ric resolution generally increases the probability that 
phenomena will be remotely sensed more accurately. 


Polarization Information 

The polarization characteristics of electromagnetic en- 
ergy recorded by a remote sensing system are an im- 
portant variable that can be used in many Earth 
resource investigations. Sunlight is polarized weakly. 
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a. 2004 color-infrared | x 1 ft. color composite. 


c. 2009 natural-color | x 1 ft. color composite. 


b. 2007 color-infrared | x 1 ft. color composite. 


Multiple-date Digital Frame Camera 
Aerial Photography of an 
Area Near White Rock, SC 


FIGURE 1-11 Digital frame camera aerial photography of residential development near White Rock, SC, collected in 2004, 
2007, and 2009. Aerial photography used with permission of Richland County GIS Division. 


However, when sunlight strikes a nonmetal object (e.g., 
grass, forest, or concrete) it becomes depolarized and 
the incident energy is scattered differentially. Generally, 
the more smooth the surface, the greater the polariza- 
tion. It is possible to use polarizing filters on passive 
remote sensing systems (e.g., aerial cameras) to record 
polarized light at various angles. It is also possible to 
selectively send and receive polarized energy using ac- 
tive remote sensing systems such as RADAR (e.g., hor- 
izontal send, vertical recetve—HV; vertical send, 
horizontal receive—VH; vertical send, vertical re- 
ceive—VV; horizontal send, horizontal recetve—HH). 
Multiple-polarized RADAR imagery is an especially 
useful application of polarized energy. 


Angular Information 


Remote sensing systems record specific angular char- 
acteristics associated with each exposed silver halide 


Radiometric Resolution 
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FIGURE 1-12 The radiometric resolution of a remote sens- 
ing system is the sensitivity of its detectors to differences in 
signal strength as they record the radiant flux reflected, 
emitted, or back-scattered from the terrain. The energy is 
normally quantized during an analog-to-digital conversion 
process to 8-, 9-, 10-bits or more. 
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c. Comparison of hourly three-dimensional plots of BRDF for smooth cordgrass (Spartina alterniflora) data 
collected at 8 a.m., 9 a.m., 12 p.m., and 4 p.m. at the boardwalk site on March 21—22, 2000, for band 624.20 nm. 


FIGURE 1-13 a) The concepts and parameters of the bidirectional reflectance distribution function (BRDF). A target is bathed 
in irradiance (dE;) from a specific Sun zenith and azimuth angle, and the sensor records the radiance (dL,) exiting the target of 
interest at a specific azimuth and zenith angle. b) The Sandmeier Field Goniometer collecting smooth cordgrass (Spartina al- 
terniflora) BRDF measurements at North Inlet, SC. Spectral measurements are made at Sun zenith angle of 0; and Sun azi- 
muth angle of @; and a sensor zenith angle of view of 0. and sensor azimuth angle of @.. A GER 3700 spectroradiometer, 
attached to the moving sled mounted on the zenith arc, recorded the amount of radiance leaving the target in 704 bands at 
76 angles. c) Hourly three-dimensional plots of BRDF data. Diagrams and photograph adapted from Schill et al., 2004. 


crystal or detector element. The angular characteristics | * the location of the suborbital or orbital remote sens- 
are a function of (Figure 1-13a): ing system and its associated azimuth and zenith 
angles. 

¢ the location in a three-dimensional sphere of the 
illumination source (e.g., the Sun for a passive sys- 
tem or the sensor itself in the case of RADAR, 
LiDAR, and SONAR) and its associated azimuth 
and zenith angles, 


* the orientation of the terrain facet (pixel) or terrain 
cover (e.g., vegetation) under investigation, and 


There is always an angle of incidence associated with 
the incoming energy that illuminates the terrain and an 
angle of exitance from the terrain to the sensor system. 
This bidirectional nature of remote sensing data collec- 
tion influences both the spectral and the polarization 
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characteristics of the at-sensor radiance, L, recorded 
by the remote sensing system. 


A goniometer can be used to measure the changes in at- 
sensor radiance, L, caused by changing the position of 
the sensor and/or the source of the illumination (e.g., 
the Sun) (Figure 1-13b). For example, Figure 1-13c 
presents three-dimensional plots of smooth cordgrass 
(Spartina alterniflora) wetland BRDF data collected at 
8 a.m.,9a.m., 12 p.m., and 4 p.m. on March 21, 2000, 
for band 624.20 nm at the Baruch Marine Lab near 
Georgetown, SC. The only thing that changed between 
observations was the Sun’s azimuth and zenith angles. 
The azimuth and zenith angles of the spectroradiome- 
ter were held constant while viewing the smooth cord- 
grass. Ideally, the BRDF plots would be identical, 
suggesting that it does not matter what time of day we 
collect the remote sensor data because the spectral re- 
flectance characteristics from the smooth cordgrass re- 
main constant. It is clear that this is not the case and 
that the time of day influences the spectral response. 
The Multiangle Imaging Spectrometer (MISR) on- 
board the Terra satellite was designed to investigate the 
BRDF phenomena. Research continues on how to in- 
corporate the BRDF information into the digital im- 
age processing system to improve our understanding of 
what is recorded in the remotely sensed imagery (e.g., 
Sandmeier, 2000; Schill et al., 2004). 


Angular information is central to the use of remote 
sensor data in photogrammetric applications. Stereo- 
scopic image analysis is based on the assumption that 
an object on the terrain is remotely sensed from two 
angles. Viewing the same terrain from two vantage 
points introduces stereoscopic parallax, which is the 
foundation for all stereoscopic photogrammetric and 
radargrammetric analysis (Wolf et al., 2013). 


Suborbital (Airborne) Remote Sensing Systems 
High-quality photogrammetric cameras mounted on- 
board aircraft continue to provide aerial photography 
for many Earth resource applications. For example, 
photogrammetric engineering firms such as Pictometry 
International, Inc. and Sanborn Map Company pro- 
vide both vertical and oblique digital aerial photogra- 
phy for many of the counties in the United States. 
These high spatial resolution datasets are crucial for 
maintaining local infrastructure including hydrologic 
networks, transportation networks, utilities, land use 
planning, zoning, tax mapping, impervious surface 
mapping, etc. 


In addition, remote sensing systems are routinely 
mounted on aircraft to provide high spatial and spec- 
tral resolution remotely sensed data. Examples include 
hyperspectral sensors such as NASA’s Airborne Visi- 
ble-Infrared Imaging Spectrometer (AVIRIS), the Ca- 


nadian Airborne Imaging Spectrometer (CASI-1500), 
and the Australian HyMap hyperspectral system dis- 
cussed in Chapter 2 and Chapter 11. These sensors can 
collect data on demand when disaster strikes if cloud- 
cover conditions permit. For example, NASA collected 
hundreds of flightlines of AVIRIS data to monitor the 
Deepwater Horizon oil spill in the Gulf of Mexico in 
2010. Unfortunately, suborbital remote sensor data are 
expensive to acquire per km?. Also, atmospheric turbu- 
lence can cause the data to have severe geometric dis- 
tortions that can be difficult to correct. 


Satellite Remote Sensing Systems 

Remote sensing systems onboard satellites provide 
high-quality, relatively inexpensive data per km. The 
United States has progressed from multispectral scan- 
ning systems (Landsat MSS launched in 1972) to more 
advanced scanning systems (Landsat 8 was launched 
on February 11, 2013). The Land Remote Sensing Pol- 
icy Act of 1992 specified the future of satellite land re- 
mote sensing programs in the United States (Jensen, 
1992). Unfortunately, Landsat 6 with its Enhanced 
Thematic Mapper did not achieve orbit when launched 
on October 5, 1993. Landsat 7 was launched on April 
15, 1999, to relieve the United States’ land remote 
sensing data gap. Unfortunately, it now has serious 
scan-line corrector problems. 


The NASA Earth Observing-1 (EO-1) mission 
launched on November 21, 2000, validated a multi- 
spectral instrument (the Advanced Land Imager— 
ALJ) that is a significant improvement over the Land- 
sat 7 ETM™ instrument; validated a hyperspectral land 
imaging instrument (Hyperion) and the unique science 
that can be performed with hyperspectral data; and 
validated the ability of a low-spatial/high-spectral reso- 
lution imager that can correct systematic errors in the 
apparent surface reflectance caused by atmospheric ef- 
fects, primarily water vapor. The breakthrough space- 
craft bus technologies that were validated were made 
available for Landsat 8 launched in 2013. 


Meanwhile, numerous other countries launched high 
and moderate and high spatial resolution panchromat- 
ic and multispectral remote sensing systems during the 
first decade of the twenty-first century (e.g., France 
[SPOT 5 and 6, Pleiades], Korea [KompsSat-2 and -3], 
and India [CartoSat-2, ResourceSat-2]). These Earth 
resource satellites are described in Chapter 2. 


Global Climate Change and Satellite Remote 
Sensing: The Earth’s physical climate subsystem is 
sensitive to changes in the Earth’s radiation balance. 
The International Geosphere—Biosphere Program (IG- 
BP) and the U.S. Global Change Research Program 
(USGCRP) both call for scientific research to describe 
and understand the interactive physical, chemical, and 


CHAPTER 1 


External 
Sun Forcing 
Functions 


ospheric Chemistry and Dynamics 
— > , 


REMOTE SENSING AND DIGITAL IMAGE PROCESSING 21 


Volcanoes 


Physical Climate System 


Ocean 
dynamics 


Water pollution 


Hydrologic Cycle 


Global moisture 


Human Activities 


Biogeochemical Cycles 


Marine 
bio- 


Ly oe . a Tropospheric a 
physics and ae 
dynamics chemistry 
i 
Terrestrial f 5 A Terrestrial 
energy and ecosystems 
moisture Soil and water 
chemistry 


Air pollution 


Land use 4 ‘ 


FIGURE 1-14 The Earth system can be subdivided into two subsystems—the physical climate system and biogeochemical cy- 
cles—that are linked by the global hydrologic cycle. Significant changes in the external forcing functions and human activities 
have an impact on the physical climate system, biogeochemical cycles, and the global hydrologic cycle. Examination of these 
subsystems and their linkages defines the critical questions that the NASA Earth Science division is attempting to answer (con- 
cept updated from Asrar and Dozier, 1994; images courtesy of NASA, the U.S. Geological Survey, and the author). 


biological processes that regulate the total Earth sys- 
tem (IGBP, 2014; USGCRP, 2014). Space-based re- 
mote sensing is an integral part of these research 
programs because it provides one of the most impor- 
tant means of observing global ecosystems consistently 
and synoptically. 


Many scientific agencies throughout the world are fo- 
cusing on monitoring the Earth to obtain accurate 
global climate change information. In this regard, NA- 
SA’s Earth Science Program’s goal is to develop a sci- 
entific understanding of Earth’s system and _ its 
response to natural or human-induced changes, and to 


improve prediction of climate, weather, and natural 
hazards. 


Early on, NASA conceptualized the remote sensing 
science conducted as part of its earth science program 
as consisting of two Earth subsystems: 1) the physical 
climate, and 2) biogeochemical cycles, linked by the 
global hydrologic cycle as shown in Figure 1-14 (Asrar 
and Dozier, 1994). 


The physical climate subsystem is sensitive to fluctua- 
tions in the Earth’s radiation balance. Many scientists 
believe that human activities have caused changes to 
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FIGURE 1-15 Monthly mean atmospheric carbon dioxide measured at Mauna Loa Observatory, Hawaii, from 1958 through 
October, 2012. Data are reported as a dry mole fraction defined as the number of molecules of carbon dioxide divided by the 
number of molecules of dry air multiplied by one million (ppm). Source of data: Scripps Institution of Oceanography and 


NOAA. 


the planet’s radiative heating mechanism that rival or 
exceed natural change. Most believe this to be due to 
an increase in atmospheric carbon dioxide. Mean 
monthly measurements of carbon dioxide at the Mau- 
na Loa Observatory, HI, since 1958 are shown in Fig- 
ure 1-15 (NOAA CO5, 2012). If this rate is sustained, it 
could result in global mean temperatures increasing 
about 0.2 to 0.5 °C per decade during the next century. 
Volcanic eruptions and the ocean’s ability to absorb 
heat may impact the projections.The following ques- 
tions are being addressed using remote sensing: 


¢ How do clouds, water vapor, and various types of 
aerosols in the Earth’s radiation and heat budgets 
change with increased atmospheric greenhouse-gas 
concentrations? 


¢ How do the oceans interact with the atmosphere in 
the transport and uptake of heat? 


¢ How do land-surface properties such as snow and 
ice cover, evapotranspiration, urban/suburban land 
use, and vegetation influence circulation? 


The Earth’s biogeochemical cycles have also been 
changed by humans (Figure 1-14). Atmospheric car- 
bon dioxide has increased by 30% since 1859, methane 
by more than 100%, and ozone concentrations in the 
stratosphere have decreased, causing increased levels of 


ultraviolet radiation to reach the Earth’s surface. The 
following questions are being addressed using remote 
sensing: 


¢ What role do the oceanic and terrestrial components 
of the biosphere play in the changing global carbon 
budget? 

¢ What are the effects on natural and managed eco- 
systems of increased carbon dioxide and acid depo- 
sition, shifting precipitation patterns, and changes 
in soil erosion, river chemistry, and atmospheric 
ozone concentrations? 


The hydrologic cycle links the physical climate and bio- 
geochemical cycles (Figure 1-14). The phase change of 
water between its gaseous, liquid, and solid states in- 
volves storage and release of latent heat, so it influenc- 
es atmospheric circulation and globally redistributes 
both water and heat (Asrar and Dozier, 1994). The hy- 
drologic cycle is the integrating process for the fluxes 
of water, energy, and chemical elements among compo- 
nents of the Earth system. Important questions being 
addressed include: 


¢ How will atmospheric variability, human activities, 
and climate change affect patterns of humidity, pre- 
cipitation, evapotranspiration, and soil moisture? 


¢ How does soil moisture vary in time and space? 


CHAPTER 1 


* Can we predict changes in the global hydrologic 
cycle using present and future observation systems 
and models? 


To address these questions, NASA launched several 
very important satellites as part of its Earth Observa- 
tion program in the late 1990s and the first decade of 
the twenty-first century. Many of the sensors continue 
to provide valuable Earth resource information. In 
particular, the EOS Terra satellite was launched on De- 
cember 18, 1999. It contained five remote sensing in- 
struments described in Chapter 2, including the: 

¢ Moderate Resolution 
(MODIS) 


* Advanced Spaceborne Thermal Emission and Reflec- 
tion Radiometer (ASTER), 


¢ Multiangle Imaging Spectro Radiometer (MISR), 


Imaging — Spectrometer 


* Clouds and the Earth’s Radiant Energy System 
(CERES), and 


¢ Measurements of Pollution in 
(MOPITT). 


The EOS Aqua satellite was launched in May, 2002 
with an additional MODIS sensor. 


the Troposphere 


More recently, NASA’s Earth Science Division is coor- 
dinating the development of a series of satellite and air- 
borne missions for long-term global observations of 
the land surface, biosphere, solid Earth, atmosphere, 
and oceans. The report titled Responding to the Chal- 
lenge of Climate and Environmental Change: NASA’s 
Plan for a Climate-Centric Architecture for Earth Ob- 
servations and Applications from Space describes NA- 
SA’s plan (NASA Goals, 2010). In particular, NASA is 
completing the development and launch of a set of a) 
Foundational missions, b) new Decadal Survey re- 
quested missions, and c) Climate Continuity missions 
(NASA Earth Science Program, 2013). 


The Foundational missions are those missions that were 
already in development at the time the National Re- 
search Council’s decadal survey was published in 2007, 
including: 


° Aquarius, 

¢ NPOESS Preparatory Project (NPP), 

¢ Landsat Data Continuity Mission (LDCM), and 
¢ Global Precipitation Measurement (GPM). 


The National Research Council completed its first 
decadal survey for Earth science titled Earth Science 
and Applications from Space: National Imperatives for 
the Next Decade and Beyond in 2007 at the request of 
NASA, NOAA, and USGS (NRC, 2007b). The report 
recommended that: “The U.S. government, working in 
concert with the private sector, academia, the public, 
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and its international partners, should renew its invest- 
ment in Earth-observing systems and restore its leader- 
ship in Earth science and applications.” The Decadal 
Survey missions guided by the 2007 decadal survey in- 
clude: 


¢ Soil Moisture Active-Passive (SMAP), 
¢ Ice, Cloud and land Elevation Satellite (ICESat-I]), 
¢ Hyperspectral Infrared Imager (HyspIRI), 


¢ Active Sensing of CO2 Emissions Over Nights, Days, 
and Seasons (ASCENDS), 


¢ Surface Water and Topography (SWOT), 


* Geostationary Coastal and Air Pollution Events 
(GEO-CAPE), and 


¢ Aerosol-Clouds-Ecosystems (ACE). 


The Climate Continuity missions include: 


¢ Orbiting Carbon Observatory-2 (OCO-2), 


¢ Stratospheric Aerosol and Gas Experiment — III 
(SAGE III), Gravity Recovery and Climate Experi- 
ment Follow-on (GRACE-FO), and 


¢ Pre-Aerosol, Clouds, and Ocean Ecosystem (PACE) 
(NASA Earth Science Program, 2013). 


Several of these missions are discussed in Chapter 2. 


Commercial Satellite Remote Sensing Data 
Providers: Commercial vendors continue to develop 
some of the most sophisticated remote sensing satel- 
lites for Earth resource applications. For example, 
GeoEye, Inc., launched GeoEye-1 on September 6, 
2008, with a 0.41 x 0.41 m panchromatic band and 
four 1.65 <x 1.65 m multispectral bands. DigitalGlobe, 
Inc., launched World View-2 on October 8, 2009, with a 
0.46 x 0.46 m panchromatic band and four 1.85 x 1.85 
m multispectral bands. Astrium, Inc., launched Pleia- 
des-1 on December 16, 2011, with four multispectral 
bands at 2 x 2 m and a panchromatic band at 0.5 x 0.5 
m. RapidEye, Inc., launched RapidEye on August 29, 
2008, with five multispectral bands at 5 x 5 m spatial 
resolution. ImageSat International, Inc., launched 
EROS-B on April 25, 2006, with a 0.7 x 0.7 m pan- 
chromatic band. SPOT Image launched SPOT 6 on 
September 9, 2012, with a 1.5 x 1.5 m panchromatic 
band and four multispectral bands at 8 x 8 m. The 
characteristics of these commercial satellite remote 
sensing systems are described in Chapter 2. 


Remote Sensing Data Analysis 
Remote sensor data are analyzed using a variety of im- 
age processing techniques (Figure 1-5), including: 


* analog (visual) image processing, and 
¢ digital image processing. 
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Analog and digital analysis of remotely sensed data 
seeks to detect and identify important phenomena in 
the scene. Once identified, the phenomena are usually 
measured, and the information is used in solving prob- 
lems (Jensen and Hodgson, 2005; Roy et al., 2014). 
Thus, both manual and digital analysis have the same 
general goals. However, the attainment of these goals 
may follow different paths. 


Human beings are adept at visually interpreting images 
produced by certain types of remote sensing devices, 
especially cameras. One could ask, “Why try to mimic 
or improve on this capability?” First, there are certain 
thresholds beyond which the human interpreter cannot 
detect “just noticeable differences” in the imagery. For 
example, it is commonly known that an analyst can 
discriminate only about nine shades of gray when in- 
terpreting continuous-tone, black-and-white photogra- 
phy. If the data were originally recorded with 256 
shades of gray, there might be more subtle information 
present in the image than the interpreter can extract vi- 
sually. Furthermore, the interpreter brings to the task 
all the pressures of the day, making the interpretation 
subjective and generally unrepeatable. Conversely, the 
results obtained by computer are repeatable (even 
when wrong!). Also, when it comes to keeping track of 
a great amount of detailed quantitative information, 
such as the spectral characteristics of a vegetated field 
throughout a growing season for crop identification 
purposes, the computer is very adept at storing and 
manipulating such tedious information and possibly 
making a more definitive conclusion as to what crop is 
being grown. Furthermore, digital image processing 
may improve efficiency and reduce the cost of expen- 
sive human labor. This is not to say that digital image 
processing is superior to visual image analysis. Rather, 
there may be times when a digital approach is better 
suited to the problem at hand. Optimum results are of- 
ten achieved using a synergistic combination of both 
visual and digital image processing. 


Analog (Visual) Image Processing 

Humans use the fundamental elements of image inter- 
pretation summarized in Figure 1-16, including gray- 
scale tone, color, height (depth), size, shape, shadow, 
texture, site, association, and arrangement. Most of 
these elements of image interpretation were first intro- 
duced by Olson (1960). The human mind is amazingly 
good at recognizing and associating these complex ele- 
ments in an image or photograph because we constant- 
ly process a) profile views of Earth features every day, 
and b) images seen in books, magazines, the television, 
and the Internet. Furthermore, we are adept at bring- 
ing to bear all the knowledge in our personal back- 
ground and collateral information. We then converge 
all this evidence to identify phenomena in images and 
judge their significance. Precise measurement of ob- 


jects (length, area, perimeter, volume, etc.) may be per- 
formed using photogrammetric techniques applied to 
either monoscopic (single-photo) or stereoscopic 
(overlapping) images. Numerous books have been writ- 
ten on how to perform visual image interpretation and 
photogrammetric measurement. 


There is a resurgence in the art and science of visual 
image interpretation as the digital remote sensor sys- 
tems provide increasingly higher spatial resolution im- 
agery. Many people are displaying high spatial 
resolution GeoEye-1 and WorldView-2 imagery on the 
computer screen and then visually interpreting the da- 
ta. The data are also often used as a base map in GIS 
projects (Jensen and Jensen, 2013). 


Digital Image Processing 

Scientists have made advances in digital image process- 
ing of remotely sensed data for scientific visualization 
and hypothesis testing. Many of the methods are sum- 
marized in Lillesand et al. (2008), Jensen et al. (2009), 
Warner et al. (2009a), Bossler et al. (2010), Prost 
(2013) and in this book. Digital image processing now 
makes use of many elements of image interpretation 
using the techniques summarized in Figure 1-16. Some 
of the major types of digital image processing include 
image preprocessing (radiometric and geometric cor- 
rection), image enhancement, photogrammetric image 
processing of stereoscopic imagery, parametric and 
non-parametric information extraction, expert system 
(e.g., decision-tree) and neural network image analysis, 
hyperspectral data analysis, and change detection (Fig- 
ure 1-5). 


Radiometric Correction: Analog and digital remote- 
ly sensed imagery may contain noise or error that was 
introduced by the sensor system (e.g., electronic noise) 
or the environment (e.g., atmospheric scattering of 
light into the sensor’s field of view). Advances have 
been made in our ability to remove these deleterious ef- 
fects through simple image normalization techniques 
and more advanced absolute radiometric calibration of 
the data to scaled surface reflectance (for optical data) 
(e.g., He and Chen, 2014). Calibrated remote sensor 
data allow imagery and derivative products obtained 
on different dates to be compared (e.g., to measure the 
change in LAI between two dates) (Liang, 2004). Fun- 
damental radiometric correction digital image process- 
ing principles are discussed in Chapter 6. 


Geometric Correction: Most analog and digital re- 
mote sensor data are now processed so that individual 
picture elements are in their proper planimetric (x, y) 
positions in a standard map projection (e.g., Shepherd 
et al., 2014). This facilitates the use of the imagery and 
derivative products in GIS or spatial decision support 
systems. 
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Fundamental Image Analysis Tasks 


* Detect, Identify, Measure 
* Solve problems 


Application of the Multi concept: 
- Multispectral - Multifrequency - Multipolarization 
- Multitemporal - Multiscale - Multidisciplinary 
| 


Use of Collateral Information 
- Literature - Laboratory spectra - Dichotomous keys - Prior probabilities 
- Field training sites - Field test sites - Soil maps - Surficial geology maps 


Analog (Visual) Digital 
Image Processing | Image Processing 


How the Elements of Image Interpretation Are 
Elements of Image Interpretation Extracted or Used in Digital Image Processing 
* Grayscale tone (black to white) * 8- to 12-bit brightness values or scaled to 
surface reflectance or emittance 
* 24-bit color look-up table display 
- Multiband RGB color composites 
- Transforms (e.g., intensity, hue, saturation) 
* Soft-copy photogrammetry, radargrammetry, 
RADAR interferometry, LIDAR, SONAR 
* Size (length, area, perimeter, volume)] + Soft-copy photogrammetry, radargrammetry, 
RADAR interferometry 
* Shape * Soft-copy photogrammetry, radargrammetry, 
interferometry, landscape ecology metrics, 
object-based image analysis (OBIA) segmentation 
¢ Texture ¢ Texture transforms, geostatistical analysis, 
landscape ecology metrics, fractal analysis 
¢ Pattern ¢ Autocorrelation, geostatistical analysis, 
landscape ecology metrics, fractal analysis 


* Color (RGB = red, green, blue) 


* Height (elevation) and depth 
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° Shadow * Soft-copy photogrammetry, radargrammetry, 


measurement from rectified images 


* Site * Contextual, expert system, neural network analysis 


* Association 
¢ Arrangement 


* Contextual, expert system, neural network analysis 
* Contextual, expert system, neural network analysis 


FIGURE 1-16 Analog (visual) and digital image processing of remotely sensed data make use of the fundamental elements of 


image interpretation. 


Image Enhancement: Images can be digitally en- 
hanced to identify subtle information in the analog or 
digital imagery that might otherwise be missed. Signifi- 
cant improvements have been made in our ability to 
contrast stretch and filter data to enhance low- and 
high-frequency components, edges, and texture in the 
imagery (e.g., Jensen and Jensen, 2013). In addition, 
the remote sensor data can be linearly and nonlinearly 
transformed into information that is more highly cor- 
related with real-world phenomena through principal 
components analysis and various vegetation indices 
(Nellis et al., 2009). 


Photogrammetry: Significant advances have been 
made in the analysis of stereoscopic remote sensor 
data obtained from airborne or satellite platforms us- 
ing computer workstations and image processing pho- 
togrammetric algorithms. Soft-copy photogrammetric 
workstations can be used to extract accurate digital el- 
evation models (DEMs) and differentially corrected 


orthophotography from the triangulated aerial pho- 
tography or imagery (Leprince et al., 2007; Wolf et al., 
2013). Large-scale mapping of building footprints, 
transportation network, hydrologic network, utilities, 
etc. is now performed almost exclusively using soft- 
copy photogrammetric and/or LiDARgrammetric data 
analysis (Jensen and Jensen, 2013). 


Parametric Information Extraction: Scientists at- 
tempting to extract land-cover information from re- 
motely sensed data now routinely specify if the 
classification is to be: 


¢ hard (sometimes referred to as crisp), with discrete 
mutually exclusive classes, or fuzzy, where the pro- 
portions of materials within pixels are extracted; 


¢ based on individual pixels (referred to as a per-pixel 
classification) or if it will use object-based image 
analysis (OBIJA) segmentation algorithms that take 
into account not only the spectral characteristics of 
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a pixel, but also the spectral characteristics of con- 
textual surrounding pixels. Thus, the algorithms 
take into account spectral and spatial information 
(e.g., Tullis and Jensen, 2003; Blaschke et al., 2014). 


Once these issues are addressed, it is a matter of deter- 
mining whether to use parametric (based on the analy- 
sis of normally distributed data), nonparametric, and/ 
or nonmetric classification techniques. The maximum 
likelihood classification algorithm continues to be a 
widely used parametric classification algorithm. Un- 
fortunately, the algorithm requires normally distribut- 
ed training data in m bands (rarely the case) for 
computing the class variance and covariance matrices. 
It is difficult to incorporate nonimage categorical data 
into a maximum likelihood classification. Fortunately, 
fuzzy maximum likelihood classification algorithms 
are now available (e.g., Shackelford and Davis, 2003; 
Liu et al., 2011). Support Vector Machine (SVM) clas- 
sification is also very effective especially when spectral 
training data consist of mixed pixels (Jensen et al., 
2009). 


Nonparametric Information Extraction: Nonpara- 
metric clustering algorithms, such as ISODATA, con- 
tinue to be used extensively in digital image processing 
research. Unfortunately, such algorithms depend on 
how the seed training data are extracted and it is often 
difficult to label the clusters to turn them into useful 
information classes. For these and other reasons there 
has been a significant increase in the development and 
use of artificial neural networks (ANN) for remote 
sensing applications. An ANN does not require nor- 
mally distributed training data. An ANN may incorpo- 
rate virtually any type of spatially distributed data in 
the classification. The only drawback is that sometimes 
it is difficult to determine exactly how the ANN came 
up with a certain conclusion because the information is 
locked within the weights in the hidden layer(s). Scien- 
tists continue to work on ways to extract hidden infor- 
mation so that the rules used can be more formally 
stated. The ability of an ANN to learn should not be 
underestimated. 


Nonmetric Information Extraction: It is difficult to 
make a computer understand and use the heuristic 
rules of thumb and knowledge that a human expert 
uses when interpreting an image. Nevertheless, there 
has been progress in the use of artificial intelligence 
(AI) to try to make computers do things that, at the 
moment, people do better. One area of AI that has 
great potential for image analysis is the use of expert 
systems that place all the information contained within 
an image in its proper context with ancillary data and 
extract valuable information. Duda et al. (2001) de- 
scribe various types of expert system decision-tree clas- 
sifiers as nonmetric. 


Parametric digital image classification techniques are 
based primarily on summary statistics such as the 
mean, variance, and covariance matrices. Decision-tree 
or rule-based classifiers are not based on inferential 
statistics, but instead “let the data speak for itself” 
(Gahegan, 2003). In other words, the data retains its 
precision and is not dumbed down by summarizing it 
through means, etc. Decision-tree classifiers can pro- 
cess virtually any type of spatially distributed data and 
can incorporate prior probabilities (McIver and Friedl, 
2002; Roberts et al., 2002; Im and Jensen, 2005). There 
are several approaches to rule creation, including: 1) 
explicitly extracting knowledge and creating rules from 
experts, 2) implicitly extracting variables and rules us- 
ing cognitive methods (Lloyd et al., 2002), and 3) em- 
pirically generating rules from observed data and 
automatic induction methods (Tullis and Jensen, 
2003). The development of a decision tree using hu- 
man-specified rules is time-consuming and difficult. 
However, it rewards the user with detailed information 
about how individual classification decisions were 
made (Zhang and Wang, 2003). 


Ideally, computers can derive the rules from training 
data without human intervention. This is referred to as 
machine-learning (Huang and Jensen, 1997). The ana- 
lyst identifies representative training areas. The ma- 
chine learns the patterns from these training data, 
creates the rules, and uses them to classify the remotely 
sensed data. The rules are available to document how 
decisions were made (Jensen et al., 2009). A drawback 
of artificial neural network and machine learning clas- 
sifiers in general is the need for a large sample size of 
training data. 


Hyperspectral: Special software is required to pro- 
cess hyperspectral data collected by imaging spectrora- 
diometers such as AVIRIS and MODIS. Viper Tools 
(Roberts, 2014), MultiSpec (Landgrebe and Biehl, 
2014), and the Environment for Visualizing Images - 
ENVI (Exelis ENVI, 2014) and others have pioneered 
the development of hyperspectral image analysis soft- 
ware. The software reduces the dimensionality of the 
data (number of bands) to a degree, while retaining the 
essence of the data. Under certain conditions the soft- 
ware can be used to compare the remotely sensed spec- 
tral reflectance curves with a library of spectral 
reflectance curves. Analysts are also able to identify the 
type and proportion of different materials within an in- 
dividual picture element using end-member spectral 
mixture analysis (Pu et al., 2008; Roberts, 2014). 


Modeling Using a GIS Approach: Remotely sensed 
data should not be analyzed in a vacuum without the 
benefit of collateral (ancillary) information such as soil 
maps, hydrology, and topography. For example, land- 
cover mapping using remotely sensed data has been 
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significantly improved by incorporating topographic 
information from digital terrain models and historical 
land-use information (e.g., Recio et al., 2011). GIS 
studies require timely, accurate updating of the spatial- 
ly distributed variables in the database that remote 
sensing can provide (Jensen and Jensen, 2013). Remote 
sensing can benefit from access to accurate ancillary 
information to improve classification accuracy and 
other types of modeling (e.g., Coops et al., 2006; Cho, 
2009; Pastick et al., 2011). Such synergy is critical if 
successful expert system and neural network analyses 
are to be performed. 


Scene Modeling: Strahler et al. (1986) describe a 
framework for modeling in remote sensing. Basically, a 
remote sensing model has three components: 1) a scene 
model, which specifies the form and nature of the ener- 
gy and matter within the scene and their spatial and 
temporal order; 2) an atmospheric model, which de- 
scribes the interaction between the atmosphere and the 
energy entering and being emitted from the scene; and 
3) a sensor model, which describes the behavior of the 
sensor in responding to the energy fluxes incident on it 
and in producing the measurements that constitute the 
image. They suggest that the problem of scene infer- 
ence, then, becomes a problem of model inversion in 
which the order in the scene is reconstructed from the 
image and the remote sensing model. For example, 
Woodcock et al. (1997) inverted the Li-Strahler Cano- 
py Reflectance Model for mapping forest structure. 
Deng et al. (2006) developed an algorithm to retrieve 
global leaf-area-index information. 


Basically, successful remote sensing modeling predicts 
how much radiant flux in certain wavelengths should 
exit a particular object (e.g., a conifer canopy) even 
without actually sensing the object (Liang, 2009). 
When the model’s prediction is the same as the sensor’s 
measurement, the relationship has been modeled cor- 
rectly. The scientist then has a greater appreciation for 
energy—matter interactions in the scene and may be 
able to extend the logic to other regions or applications 
with confidence. The remote sensor data can then be 
used more effectively in physical deterministic models 
(e.g., canopy structure, watershed runoff, net primary 
productivity, and evapotranspiration models) that are 
so important for large ecosystem modeling. (e.g., Dis- 
ney et al., 2006). 


Change Detection: Remotely sensed data obtained 
on multiple dates can be used to identify the type and 
spatial distribution of changes taking place in the land- 
scape (Green, 2011). Digital change detection algo- 
rithms can provide binary urban land cover “change/ 
no-change” information or more detailed “from-to” 
change information which identifies changes in land 
cover from forest, agriculture, etc., into urban land 
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cover such as residential housing, apartment complex- 
es or new roads (e.g., Jensen et al., 2009; Im et al., 
2011; Tsai et al., 2011). The change information pro- 
vides valuable insight into the processes at work (Jen- 
sen and Im, 2007; Purkis and Klemas, 2011). Change 
detection algorithms can be used on per-pixel and ob- 
ject-oriented (polygon) classifications. Unfortunately, 
there is still no universally accepted method of detect- 
ing change or of assessing the accuracy of change de- 
tection map products (Warner et al., 2009b). 


Information Presentation 


Information derived from remote sensor data are usu- 
ally summarized as an enhanced image, image map, or- 
thophotomap, thematic map, spatial database file, 
statistic, or graph (Figure 1-5). Thus, the final output 
products often require knowledge of remote sensing, 
cartography, GIS, and spatial statistics as well as the 
systematic science being investigated (e.g., soils, agri- 
culture, urban studies). Scientists who understand the 
rules and synergistic relationships of the technologies 
can produce output products that communicate effec- 
tively. Those who violate fundamental rules (e.g., car- 
tographic theory or database topology design) often 
produce poor output products that do not communi- 
cate effectively. 


Image maps offer scientists an alternative to line maps 
for many cartographic applications. Thousands of sat- 
ellite image maps have been produced from Landsat 
MSS (1:250,000 and 1:500,000 scale), TM (1:100,000 
scale) and AVHRR, and MODIS data. Image maps at 
scales >1:24,000 are possible using imagery with a spa- 
tial resolution of < 1 x 1 m. Because image map prod- 
ucts can be produced for a fraction of the cost of 
conventional line maps, they provide the basis for a na- 
tional map series oriented toward the exploration and 
economic development of the less-developed areas of 
the world, most of which have not been mapped at 
scales of 1:100,000 or larger. 


Remote sensor data that have been geometrically recti- 
fied to a standard map projection are becoming indis- 
pensable in most sophisticated GIS databases. This is 
especially true of orthophotomaps, which have the 
metric qualities of a line map and the information con- 
tent of an aerial photograph or other type of image. 


Unfortunately, when error is introduced into the re- 
mote sensing process it must be identified and report- 
ed. Innovations in error reduction include: 1) recording 
the lineage of the operations applied to the original re- 
mote sensor data, 2) documenting the geometric (spa- 
tial) error and thematic (attribute) error of the source 
materials, 3) improving legend design, especially for 
change detection map products derived from remote 
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FIGURE 1-17 Remote sensing Earth observation economics. The goal is to minimize the knowledge gap between the infor- 
mation delivery system, remote sensing experts, and the information consumer (user). The remote sensing—derived econom- 
ic, social, strategic, environmental, and/or political information must be cost-effective, and easy to use to achieve equilibrium 


(adapted from Miller et al., 2003). 


sensing, and 4) improved accuracy assessment. The re- 
mote sensing and GIS community should incorporate 
technologies that track all error in final map and image 
products as part of the metadata. This would result in 
more accurate information being used in the decision- 
making process. 


+ 


The National Research Council recognized that there 
is an economic system at play when remote sensor data 
are used for Earth resource management applications 
(Figure 1-17) (Miller et al., 2001). It consists of an in- 
formation delivery system with three components: data 
collection, image processing, and information consum- 
er (user). 


Earth Observation 
Economics 


The data collection system is composed of commercial 
vendors and public agencies that operate remote sens- 
ing systems. Private industry provides information at 
market value. Public agencies generally provide remote 
sensor data at the cost of filling a user request (CO- 
FUR). Remote sensing has been around since the 
1960s. There is an increasing number of experts that 
can use analog and/or digital image processing tech- 
niques to extract accurate geospatial information from 
the imagery. Then, there is the information consumer 


(user) of the remote sensing—derived information. The 
user generally needs information of economic, social, 
strategic, environmental and/or political value (NRC, 
1998). 


In order for the revenues generated by the information 
delivery system to be sufficient to support the capital 
and operating costs of the system, there must be a bal- 
ance (equilibrium) between the value of the informa- 
tion, as perceived by the user (consumer), and the 
revenue necessary to support the system (Miller et al., 
2001; 2003). The equilibrium has been achieved for air- 
borne photogrammetric and LiDAR mapping applica- 
tions for several decades. Time will tell if the balance 
between perceived value and cost can be maintained in 
the spaceborne case. Mergers are occurring. 


The equilibrium can also be impacted by remote sens- 
ing technology experts who do not have a good under- 
standing of the user information requirements. In fact, 
some remote sensing experts are often baffled as to 
why the lay consumers don’t embrace the remote sens- 
ing—derived information. What they fail to consider is 
that the consumers generally have no motivation to 
switch to remote sensing—derived information on eco- 
nomic, social, environmental, strategic, or political at- 
tributes simply because it is based on new technology. 
Furthermore, the consumers on the right side of Fig- 
ure 1-17 often have little knowledge of remote sensing 
technology or of how it is used to derive information. 
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TABLE 1-2 Selected remote sensing-related occupations and wage and employment trends defined by the U.S. Department 
of Labor Employment and Training Administration (USDOL/ETA, 2013; O*Net Online, http://online.onetcenter.org/find/ 
quick?s=remote sensing, photogrammetry, and geographic information systems). 


Code Occupation 


19-2099.01 Remote Sensing Scientists and 


Technologists 


Wages & Employment Trends 


Median wages (2013): $44.82 hour; $93,230 annual 
Employment (2012): 30,000 employees 


Projected growth (2012-2022): Slower than average (3 to 7%) 
Projected job openings (2012-2022): 8,300 
Top industries (2012): 1. Government 


19-4099.03 Remote Sensing Technicians 


17-1021.00 Cartographers and Photogrammetrists 


Median wages (2013): $28.29 hour; $58,840 annual 

Employment (2012): 12,000 employees 

Projected growth (2012-2022): Faster than average (15 to 21%) 
Projected job openings (2012-2022): 4,900 

Top industries (2012): 1. Professional, Scientific, and Technical Services 


2. Educational Services 


Median wages (2013): $21.25 hour; $44,200 annual 

Employment (2012): 64,000 employees 

Projected growth (2012-2022): Average (8 to 14%) 

Projected job openings (2012-2022): 31,600 

Top industries (2012): 1. Government 

2. Professional, Scientific, and Technical Services 


2. Government 


15-1199.04 Geographic Information Scientists and 


Technologists 


Median wages (2013): $39.59 hour; $82,340 annual 
Employment (2012): 206,000 employees 

Projected growth (2012-2022): Slower than average (3 to 7%) 
Projected job openings (2012-2022): 40,200 

Top industries (2012): 1. Government 


2. Professional, Scientific, and Technical Services 


Notes: 


Projected Growth is the estimated change in total employment over the projection period (2012-2022). 

Projected Job Openings represent openings due to growth and replacement. 

Industries are broad groups of businesses or organizations with similar activities, products, or services. Occupations are considered part 
of an industry based on their employment. There are hundreds of specific occupations within the “Government” and within the “Profes- 
sional, Scientific, and Technical Service” industries. Detailed occupation lists can be viewed at http://www.onetonline.org/. 


Miller et al. (2001; 2003) suggest that this situation can 
create a knowledge gap between the remote sensing ex- 
perts and the information consumers (1.e., the users) 
(Figure 1-17). Bridging the gap is mandatory if we are 
to use remote sensing to solve Earth resource manage- 
ment problems. It is unlikely that the user community 
can devote the time to learn the physics of remote sens- 
ing and methods of analog and/or digital image pro- 
cessing and GIS modeling necessary to produce useful 
information. Conversely, there is considerable interest 
on the technology side of the problem to build a com- 
munication bridge. Therefore, one way to decrease the 
size of the knowledge gap is for the remote sensing 
technologists to work more closely with the user com- 
munities to understand their requirements. This will 
lead to more useful remote sensing—derived informa- 
tion of value to the user communities. 


Advances in remote sensing image delivery systems by 
commercial firms such as Google’s Google Earth and 
Microsoft’s Bing Maps are having a significant impact 
on the public’s use and appreciation of remote sensor 
data. 


Remote Sensing/Digital 
Image Processing Careers 
in the Public and Private 
Sectors 


Before forging ahead into the next chapter, it is useful 
to take a brief look at why knowledge of remote sens- 
ing and digital image processing might be of value to 
you. Currently, there are many GIScience employment 
opportunities in the public or private sectors. The 
O*NET program is the nation’s primary source of geo- 
spatial occupational information (USDOL/ETA, 
2014). The O*NET database contains information on 
>900 standardized occupation-specific descriptors (Ta- 
ble 1-2). It is available to the public at no cost and it is 
continually updated by surveying a broad range of 
workers from each occupation. A search of the O*Net 
database using the keywords “remote sensing, photo- 
grammetry, and geographic information systems” 
identified several remote sensing—related occupations 
and their wage and employment trends through 2022. 
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Just a few of the remote sensing, photogrammetry, and 
GIS-related occupations are listed in Table 1-2. 


It is clear from Table 1-2 that a) in 2012 there were 
30,000 remote sensing scientists and technologists and 
64,000 remote sensing technicians, b) there is signifi- 
cant demand for more people in the field from 2012 to 
2022 (>39,000), with a projected growth in remote 
sensing-related occupations from 3% to 14%, and c) 
the remote sensing scientist occupations pay twice as 
well as the remote sensing technician positions. There 
were more than 206,000 GIS scientists and technolo- 
gists in 2012 with a projected demand of 40,200 from 
2012 to 2022. The top industries associated with the re- 
mote sensing—related occupations are “Government” 
“Educational Services,” or “Professional, Scientific, or 
Technical Services.” 


A recent report titled Future US. Workforce for Geo- 
spatial Intelligence (NRC, 2013) predicts that people 
trained in remote sensing, digital image processing, 
photogrammetry, and other GIScience-related fields 
will continue to be in demand in the National Geospa- 
tial-Intelligence Agency (NGA) and the military. 


Remote Sensing/Digital Image Processing 
Careers in the Public Sector 

Public remote sensing and digital image processing em- 
ployment will provide you with many opportunities to 
use remote sensing science for the public good and to 
interact with citizens on a regular basis. Public employ- 
ees trained in remote sensing and digital image pro- 
cessing help monitor and protect natural resources, 
plan our cities, and monitor and maintain our infra- 
structure (roads, waterways, etc.). National Homeland 
Security and defense-related agencies such as the 
NGA, will likely continue to hire a great number of 
GIScience professionals in coming years (NRC, 2013). 


Many people trained in remote sensing and digital im- 
age processing are drawn to public sector employment 
because of the perceived stability of employment. Al- 
though specific duties will vary by job, important as- 
pects of remote sensing-related jobs will continue to 
be: a) procuring remote sensor data, b) analyzing re- 
mote sensor data to extract meaningful information, c) 
collecting in situ data in support of remote sensing mis- 
sions, d) processing the remote sensing—derived infor- 
mation with other geospatial information in a GIS to 
perform predictive modeling, and e) maintaining large 
geospatial databases. 


Working for a public agency will require you to provide 
regular reports/presentations to governmental organi- 
zations, such as city/county councils, planning agen- 
cies, public interest groups, etc. Therefore, verbal, 
written, and graphic communication skills are indis- 


pensable, particularly when describing the results and 
significance of remote sensing-—related studies. 


Unless “classified” as secret by the government, most 
of the data and projects that you work on in the public 
sector are available to the general public via the Free- 
dom of Information Act. Public employees (except 
those in sensitive defense or homeland security related 
occupations) are generally allowed to publish the re- 
sults of their remote sensing—related studies in popular 
or peer-reviewed literature. GIScience professionals 
working in public colleges and universities are expected 
to publish their research in peer-reviewed journals. 


Remote Sensing/Digital Image Processing 
Careers in the Private Sector 

Working for a private remote sensing and/or photo- 
grammetric engineering firm will most likely require 
you to a) have a thorough knowledge of remote sensing 
and digital image processing, b) prepare proposals, of- 
ten in response to a government or commercially spon- 
sored Request for Proposal (RFP), and c) work on 
very specific projects with relatively tight schedules. 
Many private-sector employees are able to propose 
very specific remote sensing—related projects that are of 
great interest to them. To some this is an important 
private-sector consideration. Once again, verbal, writ- 
ten, and graphic communication skills are important. 


Remote sensing-related data analysis results created 
while working in the private sector may be proprietary 
to the company or firm. You may or may not be al- 
lowed to publish your results in popular or refereed 
journal literature. Proprietary methods, procedures, 
and patents are the life-blood of many commercial 
firms who must maintain a competitive intellectual and 
economic advantage to survive. 


To help prepare you to perform well in either the pub- 
lic or private sectors, you should have a sound knowIl- 
edge of remote sensing and digital image processing 
fundamental principles and be proficient using the 
most widely adopted digital image processing, soft- 
copy photogrammetry, and GIS software. 


¥ 


Remote sensing is used for numerous applications such 
as medical imaging (e.g., x-raying a broken arm), non- 
destructive evaluation of products on an assembly line, 
and analysis of Earth resources. This book focuses on 
the art and science of applying remote sensing for the 
extraction of useful Earth resource information (Fig- 
ure 1-18). Earth resource information is defined as any 


Earth Resource Analysis 
Perspective 
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Organization of 
Introductory Digital Image Processing 


Chapter 1. 
Remote Sensing & Digital Image Processing 
¢ Statement of the problem 
* Select appropriate logic 
* Select appropriate model 


Chapter 9. 
Information Extraction using 
Pattern Recognition 
* Classification scheme 
+ Feature selection 
* Supervised/unsupervised classification 
* Fuzzy classification 
Chapter 3. * Object-based classification 
Digital Image Processing Hardware 
and Software 


¢ Configure software and hardware 
Chapter 10. 


Information Extraction using 
Artificial Intelligence 
Chapter 4. * Expert systems 
Image Quality Assessment and + Artificial neural networks 
Statistical Evaluation 
¢ Univariate and multivariate statistics 
* Geostatistics 
Chapter 11. 
Information Extraction using 
Hyperspectral Image Analysis 
* Radiometric/geometric correction 
¢ Dimensionality reduction 
¢ Endmember determination 
* Mapping 
¢ Matched filtering 


Chapter 5. 
Display Alternatives and Scientific Visualization 
* 8-bit B&W and color look-up tables 
¢ 24-bit color look-up tables 


Chapter 13. 
Accuracy Assessment 
¢ Methods of accuracy assessment 
¢ Sampling design 
* Univariate/multivariate statistical analysis 


FIGURE 1-18 This book is organized according to the remote sensing process (see Figure 1-5). 
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information concerning terrestrial vegetation, soils, 
minerals, rocks, water, and urban infrastructure as well 
as certain atmospheric characteristics. Such informa- 
tion may be useful for modeling the global carbon cy- 
cle, the biology and biochemistry of ecosystems, 
aspects of the global water and energy cycle, climate 
variability and prediction, atmospheric chemistry, 
characteristics of the solid Earth, population estima- 
tion, and monitoring land-use change and natural haz- 
ards (e.g., Mulder and Coops, 2014). 


¥ Book Organization 


This book is organized according to the remote sensing 
process (Figures 1-5 and 1-18). The analyst first defines 
the problem and identifies the data required to accept 
or reject research hypotheses (Chapter 1). If a remote 
sensing approach to the problem is warranted, the ana- 
lyst evaluates several data acquisition alternatives, in- 
cluding traditional aerial photography, multispectral 
scanners, and linear and area array multispectral and 
hyperspectral remote sensing systems (Chapter 2). For 
example, the analyst may digitize existing aerial pho- 
tography or obtain the data already in a digital format 
from numerous public agencies (e.g., Landsat 8 data 
from the U. S. Geological Survey) or commercial firms 
(e.g., DigitalGlobe, Inc., RapidEye, Inc., Astrium Geo- 
information Services, Inc., Itres, Inc., HyVista, Inc.). If 
the analysis is to be performed digitally, an appropriate 
digital image processing system is configured (Chapter 
3). The image analysis begins by first computing fun- 
damental univariate and multivariate statistics of the 
raw digital remote sensor data (Chapter 4). The imag- 
ery is then viewed on a computer screen or output to 
various hard-copy devices to analyze image quality 
(Chapter 5). 


The imagery is then often preprocessed to reduce envi- 
ronmental and/or remote sensor system distortions. 
This preprocessing usually includes radiometric and 
geometric correction (Chapters 6 and 7). Various im- 
age enhancements may then be applied to the corrected 
data for improved visual analysis or as input to further 
digital image processing (Chapter 8). Thematic infor- 
mation may then be extracted from the imagery using 
pattern recognition, artificial intelligence, and/or hy- 
perspectral image analysis techniques (Chapters 9-11). 
Multiple dates of imagery can be analyzed to identify 
change that provides insight into the processes at work 
(Chapter 12). Methods of assessing the accuracy of re- 
mote sensing—derived thematic map products are sum- 
marized in Chapter 13. Sources of remote sensing 
imagery and other selected geospatial information are 
summarized in the Appendix. 
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2 Remote Sensing Data Collection 


Remotely sensed data must be in a digital format to 
perform digital image processing. There are two funda- 
mental ways to obtain digital imagery: 


1. acquire remotely sensed imagery in an analog 
format (often referred to as hard-copy) and then 
convert it to a digital format through the process of 
digitization, or 

2. acquire remotely sensed imagery already in a 
digital format, such as that obtained by the 
Landsat 7 Enhanced Thematic Mapper Plus 
(ETM”“) sensor system, Landsat 8, Pleiades, or 
WorldView-3. 


Overview 


The chapter first introduces digital image terminology 
and how analog imagery is converted into digital imag- 
ery using digitization. It then provides detailed infor- 
mation about public and commercial satellite remote 
sensing systems (e.g., Landsat 7 and 8, SPOT 5-7, 
NPOESS Preparatory Project, Pleiades, GeoEye-1, 
WorldView-2 and -3, RADARSAT-1 and suborbital 
remote sensing systems that are often used to collect 
panchromatic, multispectral and hyperspectral imag- 
ery (e.g., Pictometry, AVIRIS, CASI 1500). The chap- 
ter provides details about the sensor system hardware, 
including: scanning systems, linear arrays, area arrays, 
and digital frame cameras. Formats for storing digital 
remote sensor data are described, including: band se- 
quential, band interleaved by line, and band inter- 
leaved by pixel. 


Analog (Hard-Copy) Image 
Digitization 


Scientists and laypersons often obtain analog (hard- 
copy) remote sensor data that they desire to analyze 
using digital image processing techniques. Analog aeri- 
al photographs are, of course, ubiquitous because 
much of the world has been photographed many times. 
Occasionally scientists encounter hard-copy thermal- 
infrared or active microwave (RADAR) imagery. To 
convert analog imagery into digital imagery, the person 
performing the digital image processing must first un- 
derstand digital image terminology. 


Digital Image Terminology 


Digital remote sensor data are usually stored as a ma- 
trix (array) of numbers. Each digital value is located at 
a specific row (7) and column (j) in the matrix (Figure 
2-1). A pixel is defined as “a two-dimensional picture 
element that is the smallest non-divisible element of a 
digital image.” Each pixel at row (i) and column (j) in 
the image has an original brightness value (BV) associ- 
ated with it (some scientists use the term digital number 
[DN] value). The dataset may consist of n individual 
bands (k) of multispectral or hyperspectral imagery. 
Thus, it is possible to identify the brightness value of a 
particular pixel in the dataset by specifying its row (i), 
column (/), and band (k) coordinate, 1e., BV; ;, . It is 
important to understand that the n bands are all geo- 
metrically registered to one another. Therefore, a road 
intersection in band | at row 3, column 3 (1.e., BV3 3 7) 
should be located at the same row and column coordi- 
nate in the fourth band (i.¢., BV3 3 4). 
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Digital Image Terminology and Radiometric Resolution 


Brightness value Associated 
Columns (/) range (8-bit) (9-bit) (11-bit) grayscale 
Pees 255 white 511 white 2047 4- white 
Rows (7) 
2 
3 127 gray 255 gray 1023 gray 
74 
Bands (k) 0) black 0 black 0 black 


s 
vo 


x-axis (j columns) 


Picture element (pixel) at location row 3, column 3, band 1 
has a brightness value of 20, i.e., BV3 3 7 = 20. 


FIGURE 2-1 Digital remote sensor data are stored in a matrix (raster) format. Picture element (pixel) brightness values (BV) 
are located at row i, column j, and band k in the multispectral or hyperspectral dataset. The digital remote sensor brightness 
values are often stored as 8-bit bytes with values ranging from 0 to 255. However, several image digitization systems and 
some remote sensing systems now routinely collect 10-, 11-, or 12-bit data. 


In an analog image, the brightness value (BV) is a sur- 
rogate for the density (D) of the light-absorbing silver 
or dye deposited at a specific location. The density of a 
negative or positive transparency film is measured us- 
ing a densitometer. There are several types of densitom- 
eters, including flatbed and drum microdensitometers, 
video densitometers, and linear or area array charge- 
coupled-device densitometers. 


Microdensitometer Digitization 


The characteristics of a typical flatbed microdensitome- 
ter are shown in Figure 2-2. This instrument can mea- 
sure the density characteristics of very small portions 
of a negative or positive transparency, down to just a 
few micrometers in size, hence the term microdensitom- 
eter. Basically, a known quantity of light is sent from 
the light source toward the receiver. If the light en- 
counters a very dense portion of the film, very little 
light is transmitted to the receiver. If the light encoun- 
ters a very clear portion of the film, then much of the 
light is transmitted to the receiver. The densitometer 
can output the characteristics at each i,j location in the 
photograph in terms of transmittance, opacity, or den- 
sity. The ability of a portion of a developed film to pass 
light is called its transmittance (t ; 5 ). A black portion 
of the film may transmit no light, while a clear portion 
of the film may transmit almost 100 percent of the inci- 
dent light. Therefore, the transmittance at location i,j 
in the photograph is: 


_ light passing through the film 


total incident light en) 


i,j 


There is an inverse relationship between transmittance 
and the opacity of an area on the film. An area in the 
film that is very opaque does not transmit light well. 
Opacity (Oj ; ) is the reciprocal of transmittance: 


2 


i,J 47 : 


(2.2) 


Transmittance and opacity are two good measures of 
the darkness of any portion of a developed negative. 
However, psychologists have found that the human vi- 
sual system does not respond linearly to light stimula- 
tion, but rather we respond logarithmically. Therefore, 
it is common to use density (Dj; ;), which is the com- 
mon logarithm of opacity, as the digitization measure 
of choice: 


1 
ee = 108 199%, = log(+) 


i,j 


(2.3) 


If 10% of the light can be transmitted through a film at 
a certain i,j location, transmittance is 1/10, opacity is I/ 
0.10 or 10, and density is the common logarithm of 10 
or 1.0. 


As previously mentioned, the amount of light recorded 
by the receiver in densitometric units is commonly con- 
verted into a digital brightness value (BV; ;;.), which 
refers to the location in the photograph at row i, col- 
umn j, and band k. At the end of each scan line, the 
light source steps in the y-direction some Ay to scan 
along a line contiguous and parallel to the previous 
one. As the light source is scanned across the image, 
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Characteristics of a Flatbed Optical-Mechanical Microdensitometer 


Receiver Filter wheel (Red, Green, and Blue used 


when scanning color maps or photographs) 


Brightness value of the digitized Positive 
photograph or map, BV; ik transparency 


ra 


latbed 
surface 


FIGURE 2-2 Schematic of a flatbed microdensitometer. The hard-copy remotely sensed imagery (usually a positive transpar- 
ency) is placed on the flatbed surface. A small light source (perhaps 10 pm in diameter) is moved mechanically across the flat 
imagery in the x-direction, emitting a constant amount of light. On the other side of the imagery, a receiver measures the 
amount of energy that passes through. When one line scan is complete, the light source and receiver step in the y-direction 
some Ay to scan an area contiguous and parallel to the previous scan line. The amount of energy detected by the receiver 
along each scan line is eventually changed from an electrical signal into a digital value through an analog-to-digital (A-to-D) 
conversion. After the entire image has been scanned in this fashion, a matrix (raster) of brightness values is available for digi- 
tal image processing purposes. A color filter wheel may be used if the imagery has multiple dye layers that must be digitized. 
In this case the imagery is scanned three separate times using three different filters to separate it into its respective blue, 
green, and red components. The resulting three matrices should be in near-perfect registration, representing a multispectral 


digital dataset (from Jensen and Jensen, 2013). 


the continuous output from the receiver is converted to 
a series of discrete numerical values on a pixel-by-pixel 
basis. This analog-to-digital (A-to-D) conversion pro- 
cess results in a matrix of values that is usually record- 
ed in 8-bit bytes (values ranging from 0 to 255). These 
data are then stored on disk or tape for future analysis. 


Table 2-1 summarizes the relationship between digitiz- 
er scanning spot size (IFOV) measured in dots-per- 
inch (DPI) or micrometers (um) and the pixel ground 
resolution at various scales of aerial photography or 
imagery. The algorithms for converting from DPI to 
um and vice versa are also provided. Please remember 
that scanning imagery at spot sizes <12 um may result 
in noisy digitized data because the spot size approach- 
es the dimension of the film’s silver halide crystals. 


A simple black-and-white photograph has only a single 
band, k = 7. However, we may need to digitize color 
photography. In such circumstances, we use three spe- 
cially designed filters that determine the amount of 
light transmitted by each of the dye layers in the film 
(Figure 2-2). The negative or positive color transparen- 
cy is scanned three times (k = /, 2, and 3), each time 


with a different filter. This extracts spectral informa- 
tion from the respective dye layers found in color and 
color-infrared aerial photography and results in a co- 
registered three-band digital dataset for subsequent 
image processing. 


Rotating-drum optical-mechanical scanners digitize 
the imagery in a different fashion (Figure 2-3). The 
film transparency is mounted on a glass rotating drum 
so that it forms a portion of the drum’s circumference. 
The light source is situated in the interior of the drum. 
The y-coordinate scanning motion is provided by the 
rotation of the drum. The x-coordinate is obtained by 
the incremental translation of the source-receiver op- 
tics after each drum revolution. 


Flatbed and rotating drum microdensitometers yield 
the most accurate raster digitization of maps and imag- 
es. Microdensitometers are often found at labs that 
conduct research to very exacting standards (e.g., soft- 
copy photogrammetry) or at service companies that 
specialize in providing customers with high-quality ras- 
ter digitization. 
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Characteristics of a Rotating-Drum Optical-Mechanical Microdensitometer 


Receiver 


Light source 
inside drum 


x-axis (7 columns) 


Filter wheel (Red, Green, and Blue used 
when scanning color maps or photographs) 


Positive 
transparency 
image on drum 


(i rows) 


FIGURE 2-3 The rotating-drum, optical-mechanical scanner works on exactly the same principle as the flatbed microdensi- 
tometer except that the remotely sensed data are mounted on a rotating drum so that they form a portion of the drum’s cir- 
cumference. The light source is situated in the interior of the drum, and the drum is continually rotated in the y-direction. The 
x-coordinate is obtained by the incremental translation of the source-receiver optics after each drum revolution. Some micro- 
densitometers can write to film as well as digitize from film. In such cases the light source (usually a photodiode or laser) is 
modulated such that it exposes each picture element according to its brightness value. These are called film-writers and pro- 
vide excellent hard-copy of remotely sensed data (from Jensen and Jensen, 2013). 


Video Digitization 


It is possible to digitize hard-copy imagery by sensing 
it through a video camera and then performing an ana- 
log-to-digital conversion on the 525 columns by 512 
rows of data that are within the standard field-of-view 
(as established by the National Television System 
Committee). Video digitizing involves freezing and 
then digitizing a frame of analog video camera input. 
A full frame of video input can be read in approximate- 
ly 1/60 sec. A high-speed analog-to-digital converter, 
known as a frame grabber, digitizes the data and stores 
it in frame buffer memory. The memory is then read by 
the host computer and the digital information is stored 
on disk or tape. 


Video digitization of hard-copy imagery is performed 
very rapidly, but the results are not always useful for 
digital image processing purposes due to: 


¢ differences in the radiometric sensitivity of various 
video cameras, and 


* vignetting (light fall-off) away from the center of the 
image being digitized. 


These characteristics can affect the spectral data ex- 
tracted from the imagery. Also, any geometric distor- 
tion in the vidicon optical system will be transferred to 
the digital remote sensor data, making it difficult to 


edge-match between adjacent images digitized in this 
manner. 


Linear and Area Array Charge- 
Coupled-Device Digitization 


Advances in the personal computer industry have 
spurred the development of flatbed, desktop linear ar- 
ray digitizers based on linear array charge-coupled de- 
vices (CCDs) that can be used to digitize hard-copy 
negatives, paper prints, or transparencies at 50 to 6,000 
pixels per inch (Figure 2-4a—c). The hard-copy photo- 
graph is placed on the glass. The digitizer optical sys- 
tem illuminates an entire line of the hard-copy 
photograph at one time with a known amount of light. 
A linear array of detectors records the amount of light 
reflected from or transmitted through the photograph 
along the array and performs an analog-to-digital con- 
version. The linear array is stepped in the y-direction, 
and another line of data is digitized. 


It is possible to purchase useful desktop color scanners 
for less than $200. Many digital image processing labo- 
ratories use these inexpensive desktop digitizers to con- 
vert hard-copy remotely sensed data and maps into a 
digital format. Desktop scanners provide surprisingly 
good spatial precision. An optional “transilluminator” 
can be purchased for back-lighting any transparency 
that needs to be scanned. Unfortunately, most desktop 


CHAPTER 2. REMOTE SENSING DATA COLLECTION 41 


TABLE 2-1 Relationship between digitizer detector instantaneous-field-of-view (IFOV) measured in dots-per-inch (DPI) or mi- 
crometers (11m), and the pixel ground resolution at various scales. 


Digitizer Detector IFOV Pixel Ground Resolution at Various Scales (in meters) 
Dots-per-inch Micrometers 1:40,000 1:20,000 1:9,600 1:4,800 1:2,400 1:1,200 

(DPI) (um) 
100 254.00 10.16 5.08 2.44 1.22 0.61 0.30 
200 127.00 5.08 2.54 122 0.61 0.30 0.15 
300 84.67 3.39 1.69 0.81 0.41 0.20 0.10 
400 63.50 2.54 127 0.61 0.30 0.15 0.08 
500 50.80 2.03 1.02 0.49 0.24 0.12 0.06 
600 42.34 1.69 0.85 0.41 0.20 0.10 0.05 
700 36.29 1.45 0.73 0.35 0.17 0.09 0.04 
800 Sile75 127 0.64 0.30 0.15 0.08 0.04 
900 28.23 edd 0.56 0.27 0.14 0.07 0.03 
1000 25.40 1.02 0.51 0.24 0.12 0.06 0.03 
1200 2a, 0.85 0.42 0.20 0.10 0.05 0.03 
1500 16.94 0.67 0.34 0.16 0.08 0.04 0.02 
2000 12.70 0.51 0.25 0.12 0.06 0.03 0.02 
3000 8.47 0.33 0.17 0.08 0.04 0.02 0.01 
4000 6.35 0.25 0.13 0.06 0.03 0.02 0.008 

Useful Scanning Conversions: 

DPI = dots per inch; um =micrometers; |= inches; M = meters 

From DPI to micrometers: pm = (2.54 / DPI) 10,000 

From micrometers to DPI: DPI = (2.54 / um) 10,000 

From inches to meters: M = | x 0.0254 

From meters to inches: | = M x 39.37 

Computation of Pixel Ground Resolution: 

PM = pixel size in meters; PF = pixel size in feet; S = photo or map scale factor 

Using DPI: PM = (S/DPI)/39.37 PF = (S/DPI)/12 

Using micrometers: PM = (S x um) 0.000001PF = (S x um) 0.00000328 

For example, if a 1:6,000 scale aerial photograph is scanned at 500 DPI, the pixel size will be (6000/500)/39.37 = 0.3048 meters per 

pixel or (6000/500)/12 = 1.00 foot per pixel. If a 1:9,600 scale aerial photograph is scanned at 50.8 tm, the pixel size will be 

(9,600 x 50.8)(0.000001) = 0.49 meters or (9,600 x 50.8)(0.00000328) = 1.6 feet per pixel. 


scanners are designed for 8.5 x 14 in. originals, and 
most aerial photographs are 9 x 9 in. Similarly, most 
hard-copy maps are greater than 8.5 x 14 in. Under 
such conditions, the analyst must digitize the 9 x 9 in. 
photograph in two multiple sections (e.g., 8.5 x 9 in. 


and 0.5 x 9 in. for a 9 X 9 in. aerial photograph) and 
then digitally mosaic the two pieces together. The mo- 
saicking process can introduce both geometric and ra- 
diometric error. Therefore, it is better to use as high- 
quality and as large a digitizer as possible to minimize 
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Linear Array CCD 


|~<— 3,000 photosites —»>| 


Area Array CCD 


Linear Array CCD Flatbed Scanner 


b. 


FIGURE 2-4 a) A linear charge-coupled device (CCD) containing 3,000 photosites. b) An oversized flatbed scanner based on 
linear array CCD technology capable of digitizing hard-copy maps and aerial photographs that are up to 12 x 16 in. It is very 
useful for digitizing 9 x 9 in. aerial photographs in a single pass. c) An example of an area array CCD used in an area-array 


scanner (from Jensen and Jensen, 2013). 


the amount of mosaicking required. A high-quality 12 
x 16 in. scanner is shown in Figure 2-4b. 


Some digitizing systems utilize area array CCD tech- 
nology (Figure 2-4c). These systems scan the film (the 
original negative or positive transparency) as a series 
of rectangular image segments or tiles. Radiometric 
calibration algorithms are then used to compensate for 
uneven illumination encountered in any of the tile re- 
gions. When scanning a color image, the scanner stops 
on a rectangular image section and captures that infor- 
mation sequentially with each of the color filters (blue, 
green, red) before it moves to another section. 


When scanning black-and-white or color images, it is 
good practice to use special Gray Scale and Color 
Control Patches such as those shown in Figure 2-5ab. 
The U. S. National Archives and Records Administra- 
tion (NARA) recommends that these cards be laid next 
to the map or image that is to be digitized (Puglia et 
al., 2004). After digitizing, the analyst looks at the 


quality of the red, green, and blue (RGB) values of the 
white, gray, and black parts of the grayscale or color 
control patches on the computer screen. If the values 
of the RGB values for the white, gray, and black test 
areas lie within the “aimpoint” range, then it is likely 
that the digitization has been successful. If the values 
fall outside the aimpoint range, then adjustments 
should be made and the image or map should be redig- 
itized until aimpoint values are achieved. 


Digitized National Aerial 
Photography Program (NAPP) Data 


The National Aerial Photography Program (NAPP) 
was initiated in 1987 as a replacement for the National 
High Altitude Aerial Photography (NHAP) Program. 
The objective of the National Aerial Photography Pro- 
gram is to acquire and archive photographic coverage 
of the coterminous United States at 1:40,000 scale us- 
ing color-infrared or black-and-white film. The pho- 
tography is acquired at an altitude of 20,000 ft. above 
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Gray Scale and Color Control Patches 
Placed Adjacent to Photographs or Maps to be Scanned 


KODAK Gray Scale 
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FIGURE 2-5 a) A Gray Scale card is placed adjacent to a black-and-white aerial photograph to be scanned. The goal is to 
have the RGB values in the digital output file match as close as possible the aimpoint RGB values specified in the table for 
white, mid, and black points. b) The Color Control Patch card is placed adjacent to the color aerial photograph to be scanned 
(courtesy of U.S. National Archives and Records Administration; Puglia et al., 2004; Jensen and Jensen, 2013). 


ground level (AGL) with a 6-in. focal-length metric 
camera. The aerial photography is acquired ideally on 
a five-year cycle, resulting in a nationwide photograph- 
ic database that is readily available through the EROS 
Data Center in Sioux Falls, SD, or the Aerial Photog- 
raphy Field Office in Salt Lake City, UT. 


High spatial resolution NAPP photography represents 
a wealth of information for on-screen photo interpre- 
tation and can become a high-resolution basemap 
upon which other GIS information (e.g., parcel bound- 


aries, utility lines, tax data) may be overlaid after it is 
digitized and rectified to a standard map projection. 
Light (1993) summarized the optimum methods for 
converting the NAPP data into a national database of 
digitized photography that meets National Map Accu- 
racy Standards. Microdensitometer scanning of the 
photography, using a spot size of 15 um, preserves the 
27 resolvable line-pair-per-millimeter (lp/mm) spatial 
resolution in the original NAPP photography. This 
process generally yields a digital dataset that has a 
ground spatial resolution of | x 1 m, depending on 
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original scene contrast. This meets most land-cover 
and land-use mapping user requirements. 


The digitized information can be color-separated into 
separate bands of information if desired. The 15-um 
scanning spot size will support most digital soft-copy 
photogrammetry for which coordinate measurements 
are made using a computer and the monitor screen 
(Light, 1993). Because the digitized NAPP data are so 
useful as a high spatial resolution GIS basemap, many 
states have entered cost-sharing relationships with the 
U.S. Geological Survey and have their NAPP coverage 
digitized and output as digital orthophotomaps. A 
large amount of NAPP data has been digitized and 
converted into digital orthophotoquads (Light, 1996). 


Digitization Considerations 


There are some basic guidelines associated with digitiz- 
ing aerial photography or other types of hard-copy re- 
mote sensor data. First, the person who will be 
digitally processing the digitized remote sensor data 
should make the decision about what dpi to use (200 
dpi, 1,000 dpi, etc.) based on the scale of the original 
imagery and the desired spatial resolution (eg., 
1:40,000-scale aerial photography scanned at 1,000 dpi 
yields 1 x 1 m spatial resolution pixels; refer to Table 2- 
1). For example, consider the digitized NAPP photog- 
raphy of Three Mile Island shown in Figure 2-6. The 
original 1:40,000-scale panchromatic photography was 
scanned at 25 to 1,000 dpi. The U.S. Geological Survey 
typically scans NAPP photography at 1,000 dpi to pro- 
duce digital orthophotoquads. Note that for all practi- 
cal purposes it is difficult to discern visually on the 
printed page any significant difference in the quality of 
the digitized data until about 150 dpi. Then, the infor- 
mation content gradually deteriorates until at 72 to 25 
dpi, individual features in the scene are very difficult to 
interpret. It is important to remember, however, that 
when the various digitized datasets are displayed on a 
CRT screen and magnification takes place (as simulat- 
ed in the bottom nine images in Figure 2-6), it is quite 
evident that there is significantly more information 
content in the higher-resolution scans (e.g., 1,000 dpi) 
than the lower-resolution scans (e.g., 100 dpi). 


A second general principle is that when digitizing 
large-scale imagery, it is not necessary to scan at ex- 
tremely high rates (e.g., 300 — 1,000 dpi) to obtain visu- 
ally acceptable imagery. For example, large-scale 
helicopter vertical photography of vehicles in a parking 
lot in Waikiki, HI, was digitized using scan resolutions 
from 10 to 1,000 dpi (Figure 2-7). Once again, there is 
very little noticeable difference in the quality of the in- 
formation at the high scan resolutions (e.g., 1,000, 500, 
300, and 200). Note however, that even the 100-dpi and 
possibly the 72-dpi digitized images still contain signif- 


icant detailed information on the printed page because 
the original aerial photography was obtained at a very 
large scale. 


Hopefully, there is a relationship between the bright- 
ness value (BV; ;, ) or density (D;;) at any particular 
location in the digitized image and the energy reflected 
from the real-world object space (O,,,, ) at the exact lo- 
cation. Scientists take advantage of this relationship by 
1) making careful in situ observations in the field, such 
as the amount of biomass for a 1 X 1 m spot on the 
Earth located at Oxy» and then 2) measuring the 
brightness value (BV; ;,) or density of the object at 
that exact location in the photograph using a densitom- 
eter. If enough samples are located in the field and in 
the photography, it may be possible to develop a corre- 
lation between the real-world object space and the im- 
age space. This is an important use of digitized aerial 
photography. 


Digital Remote Sensor Data 
Collection 


The previous section was devoted to the digitization of 
aerial photography or other types of hard-copy re- 
motely sensed data. Digitized natural color and color- 
infrared aerial photography can be considered three- 
band multispectral datasets. Properly digitized natural 
color aerial photography can be converted to blue, 
green, and red bands of registered digital data. Digi- 
tized color-infrared aerial photography can be convert- 
ed to green, red, and near-infrared bands of digital 
data. Although these three-band multispectral datasets 
are sufficient for many applications, there are times 
when even more spectral bands located at optimum lo- 
cations throughout the electromagnetic spectrum 
would be useful for a specific application. Fortunately, 
optical engineers have developed detectors that are 
sensitive to hundreds of bands in the electromagnetic 
spectrum. The measurements made by the detectors 
are usually stored in a digital format. 


Multispectral remote sensing is defined as the collection 
of reflected, emitted, or back-scattered energy from an 
object or area of interest in multiple bands (regions) of 
the electromagnetic spectrum. Hyperspectral remote 
sensing involves data collection in hundreds of bands. 
Ultraspectral remote sensing involves data collection in 
many hundreds of bands. Most multispectral and hy- 
perspectral remote sensing systems collect data in a 
digital format. The remainder of this chapter introduc- 
es the characteristics of historical, current, and pro- 
posed multispectral and hyperspectral remote sensing 
systems. 
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Vertical Aerial Photography of Three Mile Island Scanned at Various Dots-per-inch 


p. 72 dpi. q. 50 dpi. r. 25 dpi. 


FIGURE 2-6 National Aerial Photography Program (NAPP) photography of Three Mile Island, PA, digitized at various reso- 
lutions from 1,000 to 25 dpi. The original 9 x 9 in. vertical aerial photograph was obtained on September 4, 1987, at an alti- 
tude of 20,000 ft. above ground level yielding a scale of 1:40,000. Scanning at 1,000 dpi yields 1 x 1 m pixels (refer to Table 
2-1). The upper nine images are printed at approximate contact scale while the lower nine are enlarged to demonstrate ap- 
proximate information content (NAPP photography courtesy of the U.S. Geological Survey). 
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Large-scale Vertical Aerial Photography Scanned at Various Dots-per-inch 


k. 1,000 dpi enlarged. 


j. 10 dpi. 


FIGURE 2-7 Digitized large-scale panchromatic aerial photography of vehicles in a parking lot in Waikiki, HI. The original 
photograph was obtained from a helicopter platform and then scanned at rates from 1,000 to 10 dpi. It is possible to make 
out the words “24 HR DISPATCH” on the vehicle when the photography is scanned at 1,000 dpi and magnified. 
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Sensor System 
Data Collection 


Satellite 


Direct telemetry to Earth 
or indirectly through tracking 
and data relay satellites (TDRS) 


Atmosphere 


Ground 


Visual or Digital 
Information Extraction 
* Biophysical 
¢ Land-use/land-cover 


FIGURE 2-8 An overview of the way digital remotely sensed data are transformed into useful information. The data recorded 
by the detectors are often converted from an analog electrical signal to a digital value and calibrated. Ground preprocessing 
removes geometric and radiometric distortions. This may involve the use of ephemeris or ancillary (collateral) data such as 
map x, y coordinates, a digital elevation model, etc. The data are then ready for visual or digital analysis to extract biophysical 
or land-use/land-cover information. Future sensor systems will conduct preprocessing and information extraction onboard the 
remote sensing system (Blue Marble image of the Earth courtesy of NASA Earth Observatory). 


An overview of how digital remote sensor data are 
turned into useful information is shown in Figure 2-8. 
The remote sensor system first detects electromagnetic 
energy that exits from the phenomena of interest and 
passes through the atmosphere. The energy detected is 
recorded as an analog electrical signal, which is usually 
converted into a digital value through an A-to-D con- 
version. If an aircraft platform is used, the digital data 
are simply returned to Earth. However, if a spacecraft 
platform is used, the digital data are telemetered to 
Earth receiving stations directly or indirectly via track- 
ing and data relay satellites (TDRS). In either case, it 
may be necessary to perform some radiometric or geo- 
metric preprocessing of the digital remotely sensed 
data to improve their interpretability. The data can 
then be enhanced for subsequent human visual analy- 
sis or processed further using digital image processing 
algorithms. Biophysical, land-use, or land-cover infor- 
mation extracted using either a visual or digital image 
processing approach is distributed and used to make 
decisions. 


There are a great variety of digital multispectral and 
hyperspectral remote sensing systems. It is beyond the 
scope of this book to provide detailed information on 
each of them. However, it is possible to review selected 
remote sensing systems that are or will be of significant 
value for Earth resource investigations. They are orga- 
nized according to the type of remote sensing technol- 
ogy used, as summarized in Figure 2-9, including: 


Multispectral Imaging Using Discrete Detectors 
and Scanning Mirrors 


« Landsat Multispectral Scanner (MSS) 

¢ Landsat Thematic Mapper (TM) 

¢ Landsat 7 Enhanced Thematic Mapper Plus 
(ETM*) 

« NOAA Geostationary Operational Environmental 
Satellite (GOES) 


¢ NOAA Advanced Very High Resolution Radiome- 
ter (AVHRR) 


* NPOESS Preparatory Project (NPP) 
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Multispectral Imaging Using Linear Arrays 

¢ Landsat 8 (linear array “pushbroom”) 

¢ SPOT Image, Inc. (SPOT 1-7 and Vegetation sensor) 
¢ European Space Agency (Pleiades 1A and 1B) 


¢ Indian Remote Sensing System (IRS, CartoSat, 
ResourceSat) 


¢ Korean Research Institute (KOMPSAT) 
¢ Astrium, Inc. (Sentinel-2) 


¢ NASA Terra Advanced Spaceborne Thermal Emis- 
sion and Reflection Radiometer (ASTER) 


¢ NASA Terra Multi-angle Imaging Spectroradiome- 
ter (MISR) 


e Space Imaging/GeoEye, Inc. (IKONOS, GeoEye-1) 

¢ EarthWatch/DigitalGlobe, Inc. (QuickBird, World- 
View-1, -2, -3) 

¢ ImageSat International, Inc. (EROS Al and A2) 

¢ RapidEye, Inc. 


Imaging Spectrometry Using Linear and Area 
Arrays 


« NASA Earth Observer (EO-1) Advanced Land 
Imager (ALI), Hyperion, and LEISA Atmospheric 
Corrector (LAC) 


¢ NASA Jet Propulsion Laboratory Airborne Visible/ 
Infrared Imaging Spectrometer (AVIRIS) 


¢ Itres, Inc., Compact Airborne Spectrographic 
Imager (CASI-1500) 


¢ NASA Terra Moderate Resolution Imaging Spec- 
trometer (MODIS) 


¢ NASA HyspIRI 


Airborne Digital Cameras 
* UltraCAM 


* Leica 


Satellite Analog and Digital Photographic 
Systems 


¢ NASA Space Shuttle and International Space Sta- 
tion Imagery 


Multispectral Imaging Using 
Discrete Detectors and 
Scanning Mirrors 


The collection of multispectral remote sensor data us- 
ing discrete detectors and scanning mirrors has been 
with us since the mid-1960s. Despite the technology’s 
age, several new remote sensing systems still use it. 


Earth Resource Technology Satellites 
and Landsat 1-7 Sensor Systems 


In 1967, the National Aeronautics & Space Adminis- 
tration (NASA), encouraged by the U.S. Department 
of the Interior, initiated the Earth Resource Technolo- 
gy Satellite (ERTS) program. This program resulted in 
the deployment of five satellites carrying a variety of 
remote sensing systems designed primarily to acquire 
Earth resource information. The most noteworthy sen- 
sors were the Landsat Multispectral Scanner and the 
Landsat Thematic Mapper (Table 2-2). The Landsat 
program is the United States’ oldest land-surface ob- 
servation satellite system, having obtained data since 
1972. It has had a tumultuous history of management 
and funding sources. 


The chronological launch and retirement history of the 
satellites is shown in Figure 2-10. The ERTS-1 satellite, 
launched on July 23, 1972, was the first experimental 
system designed to test the feasibility of collecting 
Earth resource data by unmanned satellites. Prior to 
the launch of ERTS-B on January 22, 1975, NASA re- 
named the ERTS program Landsat, distinguishing it 
from the Seasat oceanographic RADAR satellite 
launched on June 26, 1978. At this time, ERTS-1 was 
retroactively named Landsat 1 and ERTS-B became 
Landsat 2 at launch. Landsat 3 was launched March 5, 
1978; Landsat 4 on July 16, 1982; and Landsat 5 on 
March 1, 1984. 


The Earth Observation Satellite Company (EOSAT) 
was given control of the Landsat satellites in Septem- 
ber, 1985. Unfortunately, Landsat 6 with its Enhanced 
Thematic Mapper (ETM) (a 15 x 15 m panchromatic 
band was added) failed to achieve orbit on October 5, 
1993. Landsat 7 with its Enhanced Thematic Mapper 
Plus (ETM*) sensor system was launched on April 15, 
1999. For a detailed history of the Landsat program, 
refer to the Landsat Data User Notes published by the 
EROS Data Center; Imaging Notes published by Space 
Imaging, Inc.; the NASA Landsat 7 home page (NA- 
SA Landsat 7, 2014); Irons et al. (2012); and Loveland 
and Dwyer (2012). 


Landsats | through 3 were launched into circular or- 
bits around Earth at a nominal altitude of 919 km (570 
mi). The platform is shown in Figure 2-1la. The satel- 
lites had an orbital inclination of 99°, which made 
them nearly polar (Figure 2-11b) and caused them to 
cross the equator at an angle of approximately 9° from 
normal. The satellites orbited Earth once every 103 
minutes, resulting in 14 orbits per day (Figure 2-11c). 
This Sun-synchronous orbit meant that the orbital 
plane precessed around Earth at the same angular rate 
at which Earth moved around the Sun. This character- 
istic caused the satellites to cross the equator at the 
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Remote Sensing Systems Used to Collect Aerial Photography, 
Multispectral and Hyperspectral Imagery 


Analog Frame Camera Digital Frame Camera 
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FIGURE 2-9 Six types of remote sensing systems used for multispectral and hyperspectral data collection: a) traditional an- 
alog (film) aerial photography, b) digital frame camera aerial photography based on area arrays, c) imaging using a scanning 
mirror and discrete detectors, d) multispectral imaging using linear arrays (often referred to as “pushbroom” technology), e) 
imaging with a scanning mirror and linear arrays (often referred to as “whiskbroom” technology), and f) imaging spectrome- 
try using linear and area arrays. 
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TABLE 2-2 Landsat Multispectral Scanner (MSS) and Landsat Thematic Mapper (TM) 4 and 5 sensor system characteristics. 


240 x 240 m for band 8 


Landsat Multispectral Scanner (MSS) Landsat 4 and 5 Thematic Mapper (TM) 
Band Spectral Band Spectral 
Resolution Resolution 
(um) (um) 

ANGreen 0.5-0.6 1 Blue 0.45 - 0.52 
5 Red 0.6 -0.7 2 Green 0.52 - 0.60 
6 Near-infrared 0.7 - 0.8 3 Red 0.63 - 0.69 
7 Near-infrared 0.8-1.1 4 Near-infrared 0.76 - 0.90 
Sehihenmalllniarec! 10.4 - 12.6 5 SWIR (e175 
6 Thermal Infrared 10.40 - 12.5 

7 SWIR 2.08 — 2.35 

IFOV at nadir 79 x 79 m for bands 4 through 7 30 x 30 m for bands 1 through 5, 7 


120 x 120 m for band 6 


16 days Landsat 4, 5 


Data rate 15 Mb/s 85 Mb/s 
Quantization levels 6 bit (values from 0 to 63) 8 bit (values from 0 to 255) 
Earth coverage 18 days Landsat 1, 2, 3 16 days Landsat 4, 5 


Altitude 919 km 705 km 
Swath width 185 km 185 km 
Inclination 99° 98.2° 


a. MSS bands 4, 5, 6, and 7 were renumbered bands 1, 2, 3, and 4 on La 
b. MSS band 8 was present only on Landsat 3. 


same local time (9:30 to 10:00 a.m.) on the illuminated 
side of Earth. 


Figures 2-l1c and 2-12 illustrate how repeat coverage 
of a geographic area was acquired. From one orbit to 
the next, a position directly below the spacecraft 
moved 2,875 km (1,785 mi) at the equator as the Earth 
rotated beneath it. The next day, 14 orbits later, it was 
back to its original location, with orbit 15 displaced 
westward from orbit 1 by 159 km (99 mi) at the equa- 
tor. This continued for 18 days, after which orbit 252 
fell directly over orbit 1 once again. Landsat had the 
capability of observing the entire globe (except pole- 
ward of 81°) once every 18 days, or about 20 times a 
year. There were approximately 26 km (16 m1) of over- 
lap between successive orbits. This overlap was a maxi- 
mum at 81° North and South latitudes (about 85%) 
and a minimum at the equator (about 14%). This has 
proven useful for stereoscopic analysis applications. 


ndsats 4 and 5. 


An elegant method of determining if remote sensor 
data (e.g., Landsat MSS, Thematic Mapper, ETM”? 
are available for a specific location is to use the U.S. 
Geological Survey’s Global Visualization Viewer 
(USGS GloVis, 2014). For example, suppose we are in- 
terested in locating a Landsat Thematic Mapper image 
of Charleston, SC. We can enter the Global Visualiza- 
tion Viewer and specify WRS Path 16 and Row 37 and 
search the database as shown in Figure 2-13a. If we do 
not know the path and row designation, we could a) 
move the cursor on the regional map and place it on 
Charleston, SC, or b) input the latitude and longitude 
coordinates of Charleston, SC (33.2°N, -81°W). We 
can specify the month and year (e.g., March, 2010), 
and the amount of acceptable cloud cover (e.g., 0%). 
We can also specify whether the search should be con- 
ducted regionally as in Figure 2-13a (pixels resampled 
to 1,000 m) or locally as in Figure 2-13b (pixels resam- 
pled to 240 m). 
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Chronological Launch History of the Landsat Satellites 
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Retirement Dates (www.landsat.usgs.gov) 

Landsat | - January 6, 1978 

Landsat 2 - February 25, 1982 

Landsat 3 - September 7, 1983 

Landsat 4 - June 15, 2001 

Landsat 5 - 2012 

Landsat 6 - October 5, 1993, did not achieve orbit 
Landsat 7 - Scan Line Corrector failed May 31, 2003 
Landsat 8 - February 11, 2013 

Landsat 9 - to be launched in approximately 2018 
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a. Landsat 1 MSS 1973 |} b, Landsat 5 TM 1989) c. Landsat 7 ETM+ 2003 


FIGURE 2-10 Chronological launch, retirement history, and proposed Landsat satellites. Multiple-date Landsat images of 
Dallas-Fort Worth, TX, are provided (updated from Jensen et al., 2012). 


In the context of this section on data acquisition, we 
are interested in the type of sensors carried aloft by the 
Landsat satellites and the nature and quality of remote 
sensor data provided for Earth resource investigations. 
The most important sensors were the Multispectral 
Scanner and Landsat Thematic Mapper. 


Landsat Multispectral Scanner 

The Landsat Multispectral Scanner (MSS) was placed 
on Landsat satellites 1 through 5. The MSS multiple- 
detector array and the scanning system are shown dia- 
grammatically in Figure 2-13a. Sensors such as the 
Landsat MSS (and Thematic Mapper to be discussed) 
are optical-mechanical systems in which a mirror scans 
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the terrain perpendicular to the flight direction. While 
it scanned, it focused energy reflected or emitted from 
the terrain onto discrete detector elements. The detec- 
tors converted the radiant flux measured within each 
instantaneous-field-of-view (IFOV) in the scene into 
an electronic signal (Figure 2-13a). The detector ele- 
ments were placed behind filters that pass broad por- 
tions of the spectrum. The MSS had four sets of filters 
and detectors, whereas the TM had seven. The primary 
limitation of this approach was the short residence 
time of the detector in each IFOV. To achieve adequate 


Landsat at 
yy +— 12:30 pm 
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FIGURE 2-11 a) Nimbus-style platform used for Landsats 
1,2, and 3 and associated sensor and telecommunication 
systems. b) Inclination of the Landsat orbit to maintain a 
Sun-synchronous orbit. c) From one orbit to the next, the 
position directly below the satellite moved 2,875 km (1,785 
mi) at the equator as Earth rotated beneath it. The next 
day, 14 orbits later, it was approximately back to its original 
location, with orbit 15 displaced westward from orbit 1 by 
159 km (99 mi). This is how repeat coverage of the same 
geographic area was obtained. 


signal-to-noise ratio without sacrificing spatial resolu- 
tion, such a sensor had to operate in broad spectral 
bands of > 100 nm or use optics with unrealistically 
small ratios of focal-length to aperture (f/ stop). 


The MSS scanning mirror oscillated through an angu- 
lar displacement of +5.78° off-nadir. This 11.56° field 
of view resulted in a swath width of approximately 185 
km (115 mi) for each orbit. Six parallel detectors sensi- 
tive to four spectral bands (channels) in the electro- 
magnetic spectrum viewed the ground simultaneously: 
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FIGURE 2-12 Orbital tracks of Landsat 1, 2, or 3 during a single day of coverage. The satellite crossed the equator every 103 
minutes, during which time the Earth rotated a distance of 2,875 km under the satellite at the equator. Every 14 orbits, 24 


hours elapsed. 


0.5 — 0.6 um (green), 0.6 — 0.7 um (red), 0.7 — 0.8 pm 
(reflective infrared), and 0.8 — 1.1 um (reflective infra- 
red). These bands were originally numbered 4, 5, 6, 
and 7, respectively, because a Return-Beam-Vidicon 
(RBV) sensor system also onboard the satellite record- 
ed energy in three bands labeled 1, 2, and 3. 


When not viewing the Earth, the MSS detectors were 
exposed to internal light and Sun calibration sources. 
The spectral sensitivity of the bands is summarized in 
Table 2-2 and shown diagrammatically in Figure 2-14b. 
Note that there is spectral overlap between the bands. 


The IFOV of each detector was square and resulted in 
a ground resolution element of approximately 79 x 79 
m (67,143 fi"). The voltage analog signal from each de- 
tector was converted to a digital value using an on- 
board A-to-D converter. The data were quantized to 6 
bits with a range of values from 0 to 63. These data 
were then rescaled to 7 bits (0 to 127) for three of the 
four bands in subsequent ground processing (_.e., 
bands 4, 5, and 6 were decompressed to a range of 0 to 
127). It is important to remember that the early 1970s 
Landsat MSS data were quantized to 6-bits when com- 
paring MSS data collected in the late 1970s and 1980s, 
which were collected at 8-bits. 


The MSS scanned each line across-track from west to 
east as the southward orbit of the spacecraft provided 
the along-track progression. Each MSS scene repre- 
sents a 185 x 170 km parallelogram extracted from the 
continuous swath of an orbit and contains approxi- 
mately 10 percent overlap. A typical scene contains ap- 
proximately 2,340 scan lines with about 3,240 pixels 
per line, or about 7,581,600 pixels per channel. All four 
bands represent a dataset of more than 30 million 
brightness values. Landsat MSS images provided an 
unprecedented ability to observe large geographic ar- 
eas while viewing a single image. For example, approxi- 
mately 5,000 conventional vertical aerial photographs 
obtained at a scale of 1:15,000 are required to equal the 
geographic coverage of a single Landsat MSS image. 
This allows regional terrain analysis to be performed 
using one data source rather than a multitude of aerial 
photographs. 


Landsat Thematic Mapper (TM) 

Landsat Thematic Mapper sensor systems were 
launched on July 16, 1982 (Landsat 4), and on March 
1, 1984 (Landsat 5). The TM is an optical-mechanical 
“whiskbroom” sensor that records energy in the visi- 
ble, reflective-infrared, middle-infrared, and thermal- 
infrared regions of the electromagnetic spectrum. It 
collects multispectral imagery that has higher spatial, 
spectral, temporal, and radiometric resolution than the 


54 INTRODUCTORY DIGITAL IMAGE PROCESSING 


USGS Global Visualization Viewer used to Locate Landsat Remote Sensor Data 


Collection Resotuten Maplayers Tools File Help 


CO 0% Oste 2010319 
Olty 9 Product TM LIT 


Mar + i200 > Go 
Prev Scene | Next Scene 


Landsat 4-5 TM Scene List 


Aaa 


2000m ho Limits Set 
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FIGURE 2-13 Use of the USGS Global Visualization Viewer to locate Landsat Thematic Mapper imagery of Charleston, SC. 
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Landsat MSS. Detailed descriptions of the design and 
performance characteristics of the TM can be found at 
NASA Landsat 7 (2014). 


The Landsat 4 and 5 platform and sensor are shown in 
Figure 2-15. The Thematic Mapper sensor system con- 
figuration is shown in Figure 2-16. A telescope directs 
the incoming radiant flux obtained along a scan line 
through a scan line corrector to 1) the visible and near- 
infrared primary focal plane, or 2) the middle-infrared 
and thermal-infrared cooled focal plane. The detectors 
for the visible and near-infrared bands (1-4) are four 
staggered linear arrays, each containing 16 silicon de- 
tectors. The two middle-infrared detectors are 16 indi- 
um-antimonide cells in a staggered linear array, and 
the thermal infrared detector is a four-element array of 
mercury-cadmium-telluride cells. 


Landsat TM data have a ground-projected IFOV of 30 
x 30 m for bands | through 5 and 7. The thermal infra- 
red band 6 has a spatial resolution of 120 x 120 m. The 
TM spectral bands represent important departures 
from the bands found on the traditional MSS, also car- 
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FIGURE 2-14 a) Major components of the Landsat Multi- 
spectral Scanner (MSS) system on Landsats 1 through 5 
(Landsat 3 also had a thermal infrared band). A bank of 24 
detectors (six for each of the four bands) measured informa- 
tion from Earth from an instantaneous field-of-view of 79 x 
79 m. b) Landsat MSS bandwidths. Notice that they did not 
end abruptly, as suggested by the usual nomenclature. 


ried onboard Landsats 4 and 5. The original MSS 
bandwidths were selected based on their utility for veg- 
etation inventories and geologic studies. Conversely, 
the TM bands were chosen after years of analysis for 
their value in water penetration, discrimination of veg- 
etation type and vigor, plant and soil moisture mea- 
surement, differentiation of clouds, snow, and ice, and 
identification of hydrothermal alteration in certain 
rock types (Table 2-3). The refined bandwidths and im- 
proved spatial resolution of the Landsat TM versus the 
Landsat MSS and several other sensor systems (Land- 
sat 7 and SPOTs 1 — 4) are shown graphically in Figure 
2-17. Examples of individual bands of Landsat The- 
matic Mapper imagery of Charleston, SC, obtained in 
1994 are provided in Figure 2-18. 


The Landsat TM bands were selected to make maxi- 
mum use of the dominant factors controlling leaf re- 
flectance, such as leaf pigmentation, leaf and canopy 
structure, and moisture content, as demonstrated in 
Figure 2-19. Band | (blue) provides water-penetration 
capability. Vegetation absorbs much of the incident 
blue, green, and red radiant flux for photosynthetic 
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FIGURE 2-15 Landsat 4 and 5 platform and associated 
sensor and telecommunication systems. 


purposes; therefore, vegetated areas appear dark in 
TM band | (blue), 2 (green), and 3 (red) images, as 
seen in the Charleston, SC, Landsat TM data (Figure 
2-18). Vegetation reflects approximately half of the in- 
cident near-infrared radiant flux, causing it to appear 
bright in the band 4 (near-infrared) image. Bands 5 and 
7 both provide more detail in the wetland because they 
are sensitive to soil and plant moisture conditions. The 
band 6 (thermal) image provides limited information 
of value. 


The equatorial crossing time was 9:45 a.m. for Land- 
sats 4 and 5 with an orbital inclination of 98.2°. The 
transition from a 919 km orbit to a 705 km orbit for 
Landsats 4 and 5 disrupted the continuity of Landsats 
1, 2, and 3 MSS path and row designations in the 
Worldwide Reference System. Consequently, a separate 
WRS map is required to select images obtained by 
Landsats 4 and 5. The lower orbit (approximately the 
same as the space shuttle) also increased the amount of 
relief displacement introduced into the imagery ob- 
tained over mountainous terrain. The new orbit also 
caused the period between repetitive coverage to 
change from 18 to 16 days for both the MSS and TM 
data collected by Landsats 4 and 5. 


There was a substantial improvement in the level of 
quantization from 6- to 8-bits per pixel (Table 2-2). 
This, in addition to a greater number of bands and a 
higher spatial resolution, increased the data rate from 
15 to 85 Mb/s. Ground receiving stations were modi- 
fied to process the increased data flow. Based on the 


TABLE 2-3 Characteristics of the Landsat 4 and 5 Thematic 
Mapper (TM) spectral bands. 


Band 1: 0.45 - 0.52 ym (blue). This band provides increased 
penetration of waterbodies, as well as supporting analyses of 
land-use, soil, and vegetation characteristics. The shorter wave- 
length cutoff is just below the peak transmittance of clear wa- 
ter, and the upper-wavelength cutoff is the limit of blue 
chlorophyll absorption for healthy green vegetation. Wave- 
lengths < 0.45 um are substantially influenced by atmospheric 
scattering and absorption. 


Band 2: 0.52 — 0.60 ym (green). This band spans the region 
between the blue and red chlorophyll absorption bands and re- 
acts to the green reflectance of healthy vegetation. 


Band 3: 0.63 - 0.69 ym (red). This is the red chlorophyll ab- 
sorption band of healthy green vegetation and is useful for veg- 
etation discrimination. It is also useful for soil-boundary and 
geological-boundary delineations. This band may exhibit more 
contrast than bands 1 and 2 because of the reduced effect of 
atmospheric attenuation. The 0.69-um cutoff is significant be- 
cause it represents the beginning of a spectral region from 0.68 
to 0.75 tum, where vegetation reflectance crossovers take place 
that can reduce the accuracy of vegetation investigations. 


Band 4: 0.76 - 0.90 um (near-infrared). For reasons discussed, 
the lower cutoff for this band was placed above 0.75 um. This 
band is very responsive to the amount of vegetation biomass 
and/or leaf area present. It is useful for crop identification and 
emphasizes soil/crop and land/water contrasts. 


Band 5: 1.55 - 1.75 ym (SWIR). This band is sensitive to the 
turgidity or amount of water in plants. Such information is use- 
ful in crop drought studies and in plant vigor investigations. 
This is one of the few bands that can be used to discriminate 
among clouds, snow, and ice. 


Band 6: 10.4 — 12.5 um (thermal infrared). This band mea- 
sures the amount of infrared radiant energy emitted from sur- 
faces. The apparent temperature is a function of the emissivities 
and the true (kinetic) temperature of the surface. It is useful for 
locating geothermal activity, thermal inertia mapping for geo- 
logic investigations, vegetation classification, vegetation stress 
analysis, and soil moisture studies. The band often captures 
unique information on differences in topographic aspect in 
mountainous areas. 


Band 7: 2.08 - 2.35 um (SWIR). This is an important band for 
the discrimination of geologic rock formations. It has been 
shown to be effective for identifying zones of hydrothermal al- 
teration in rocks. 


improvements in spectral, spatial, and radiometric res- 
olution, Solomonson (1984) suggested that “it appears 
that the TM can be described as being twice as effective 
in providing information as the Landsat MSS. This is 
based on its ability to provide twice as many separable 
classes over a given area as the MSS, numerically pro- 
vide two more independent vectors in the data or dem- 
onstrate through classical information theory that 
twice as much information exists in the TM data.” 


Efforts to move the Landsat program into the commer- 
cial sector began during the Carter administration in 
1979 and resulted in 1984 legislation that charged the 
National Oceanic and Atmospheric Administration to 
transfer the program to the private sector. The Earth 
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FIGURE 2-16 Major components of the Landsats 4 and 5 Thematic Mapper sensor system. The sensor is sensitive to the sev- 
en bands of the electromagnetic spectrum summarized in Table 2-2. Six of the seven bands have a spatial resolution of 30 x 


30 m; the thermal infrared band has a spatial resolution of 120 x 
tional position. 


Observing Satellite Company (EOSAT) took over op- 
eration in 1985 and was given the rights to market 
Landsat TM data. 


Landsat 7 Enhanced Thematic Mapper Plus 


On October 28, 1992, President Clinton signed the 
Land Remote Sensing Policy Act of 1992 (Public Law 


120 m. The lower diagram depicts the sensor in its opera- 


102-555). This law authorized the procurement of 
Landsat 7 and called for its launch within 5 years of 
the launch of Landsat 6. In parallel actions, Congress 
funded Landsat 7 procurement and stipulated that 
data from publicly funded remote sensing satellite sys- 
tems like Landsat must be sold to United States gov- 
ernment agencies and their affiliated users at the cost 


58 INTRODUCTORY DIGITAL IMAGE PROCESSING 


Spatial and Spectral Resolution of Selected Landsat, SPOT, 
Astrium Pleiades, and Digital Globe World View-2 Sensor Systems 


Panchromatic 


pail Panchromatic 


Panchromatic 


| 
NIR ll 


Spatial Resolution (m) 


0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.55 


Panchromatic | | | 


DigitalGlobe (WorldView-2) 


SPOT 1, 2, and 3 
High-Resolution Visible (HRV) 


Operational Land Imager (OLID) 


| 
2 ie Landsat 8 


Landsat 7 Enhanced Thematic 
Mapper Plus (ETM*) 


Landsat Thematic Mappers (TM) 4 and 5 


Landsat Multispectral Scanner (MSS) 
«+ — 1,2,3,4,and5 


Pleiades (HR-1) 


SPOT 5 High-Resolution Visible Infrared (HRVIR) 


Thermal 
infrared 


100 m 


120m 


Mi 


1.75 2.0 2.1 2.2 2.32.4 10 11 12 


Wavelength, tum 


FIGURE 2-17 Spatial and spectral resolution of the Landsat Multispectral Scanner (MSS), Landsats 4 and 5 Thematic Mapper 
(TM), Landsat 7 Enhanced Thematic Mapper Plus (ETM*), Landsat 8 OLI, SPOTs 1, 2, and 3 High-Resolution Visible (HRV), 
SPOT 5 High-Resolution Visible Infrared (HRVIR) sensor, Pleiades HR-1, and DigitalGlobe WorldView-2 sensor systems. The 
SPOTs 4 and 5 Vegetation sensor characteristics are not shown (it consists of four 1.15 x 1.15 km bands). 


of fulfilling user requests. Unfortunately, Landsat 6 did 
not achieve orbit on October 5, 1993. 


With the passage of the Land Remote Sensing Policy 
Act of 1992, oversight of the Landsat program began 
to be shifted from the commercial sector back to the 
federal government. NASA was responsible for the de- 
sign, development, launch, and on-orbit checkout of 
Landsat 7, and the installation and operation of the 


ground system. The U.S. Geological Survey (USGS) 
was responsible for data capture, processing, and dis- 
tribution of the Landsat 7 data, mission management, 
and maintaining the Landsat 7 data archive. 


Landsat 7 was launched on April 15, 1999, into a Sun- 
synchronous orbit (Figure 2-20). Landsat 7 was de- 
signed to work in harmony with NASA’s EOS Terra 


(ts 


Landsat 5 Thematic Mapper Data of Charleston, SC 


h. Color composite bands 4,3,2 (RGB). i. Color composite bands 7,4,2 (RGB). 
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f. Band 6 (thermal infrared). 
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FIGURE 2-18 Landsat 5 Thematic Mapper data of Charleston, SC, obtained on February 3, 1994. Bands 1 through 5 and 7 
are 30 x 30 m spatial resolution. Band 6 is 120 x 120 m (images courtesy of NASA). 


satellite. It was also designed to achieve three main ob- 
jectives: 


* maintain data continuity by providing data that are 
consistent in terms of geometry, spatial resolution, 
calibration, coverage characteristics, and spectral 
characteristics with previous Landsat data; 


* generate and periodically refresh a global archive of 
substantially cloud-free, sunlit landmass imagery; 
and 

* continue to make Landsat-type data available to 
U.S. and international users at the cost of fulfilling 
user requests (COFUR) and to expand the use of 


such data for global-change research and commer- 
cial purposes. 


Landsat 7 is a three-axis stabilized satellite platform 
carrying a single Nadir-pointing instrument, the 
ETM* (Figure 2-20). The ETM” instrument is a deriv- 
ative of the Landsat 4 and 5 Thematic Mapper sensors. 
Therefore, it is possible to refer to Figure 2-16 for a re- 
view of its mirror and detector design. The ETM* was 
based on scanning technology despite the fact that lin- 
ear array “pushbroom” technology had been commer- 
cially available since the launch of the French SPOT 1 
satellite in 1986. Nevertheless, the ETM* instrument is 
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FIGURE 2-19 Progressive changes in percent reflectance for a sycamore leaf at varying oven-dry-weight moisture content. 
The dominant factors controlling leaf reflectance and the location of six of the Landsat TM bands are superimposed. 
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an excellent sensor with several notable improvements 
over its predecessors, Landsat 4 and 5. 


The characteristics of the Landsat 7 ETM”* are sum- 
marized in Tables 2-3 and 2-4. The ETM” bands 1 
through 5 and 7 are identical to those found on Land- 
sats 4 and 5 and have the same 30 x 30 m spatial reso- 
lution. The thermal infrared band 6 has 60 x 60 m 
spatial resolution (instead of 120 <x 120 m). Perhaps 
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FIGURE 2-20 Artist’s rendition 
—<* of the Landsat 7 satellite with its 
4 : Enhanced Thematic Mapper Plus 
(ETM*) sensor system (courtesy 
of NASA). 


most notable was the new 15 x 15 m panchromatic 
band (0.52 — 0.90 um). Landsat 7 ETM”® individual 
band images of San Diego, CA, are shown in Figure 2- 
21. An ETM” color composite of San Diego is shown 
in Figure 2-22. Landsat 7 is in orbit 705 km above the 
Earth, collects data in a swath 185 km wide, and can- 
not view off-Nadir. Its revisit interval is 16 days. The 
ETM* records 150 megabits of data each second. 
Landsat 7 can transmit data to ground receiving sta- 
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TABLE 2-4 Landsat Enhanced Thematic Mapper Plus (ETM*) compared with the Earth Observer (EO-1) sensors. 


Landsat 7 Enhanced Thematic Mapper Plus (ETM*) 


EO-1 Advanced Land Imager (ALI) 


Spectral Spatial Spectral Spatial 

Resolution Resolution Resolution Resolution 

Band (um) (m) at Nadir Band (um) (m) at Nadir 
1 Blue 0.450 - 0.515 30 x 30 MS-1 0.433 - 0.453 30 x 30 
2 Green 0.525 — 0.605 30 x 30 MS-1 0.450 - 0.510 30 x 30 
3 Red 0.630 — 0.690 30 x 30 MS-2 0.525 - 0.605 30 x 30 
4 Near-infrared 0.750 — 0.900 30 x 30 MS-3 0.630 — 0.690 30 x 30 
5 SWIR (less = 1167/5) 30 x 30 MS-4 0.775 — 0.805 30 x 30 
6 Thermal IR 10.40 — 12.50 60 x 60 MS-4' 0.845 — 0.890 30 x 30 
7 SWIR 2.08 - 2.35 30 x 30 MS-5’ 1.20 - 1.30 30 x 30 
8 Panchromatic 0.52 - 0.90 Sexe) MS-5 i oto nms Ae) 30 x 30 
MS-7 2.08 — 2.35 30 x 30 
Panchromatic 0.480 - 0.69 10 x 10 


EO-1 Hyperion Hyperspectral Sensor 
220 bands from 0.4 to 2.4 um at 30 x 30m 


EO-1 LEISA Atmospheric Corrector (LAC) 
256 bands from 0.9 to 1.6 um at 250 x 250 m 


Sensor Advanced Land Imager is a pushbroom radiometer. 
Technology Scanning mirror spectrometer Hyperion is a pushbroom spectroradiometer. 
LAC uses area arrays. 

Swath Width 185 km ALI = 37 km; Hyperion = 7.5 km; LAC = 185 km 
Data Rate 250 images per day @ 31,450 km? — 
Revisit 16 days 16 days 
Orbit and 705 km, Sun-synchronous 705 km, Sun-synchronous 
Inclination Inclination = 98.2° Inclination = 98.2° 

Equatorial crossing 10:00 a.m. +15 min. Equatorial crossing = Landsat 7 + 1 min 
Launch April 15, 1999 November 21, 2000 


tions at the EROS Data Center in Sioux Falls, SD, or 
to Fairbanks, AK. Landsat 7 international data may be 
acquired by retransmission using TDRS satellites or by 
international receiving stations. 


The Landsat 7 ETM” has excellent radiometric cali- 
bration, which is accomplished using partial and full 
aperture solar calibration. Ground look calibration is 
performed by acquiring images of certain Earth land- 


mass calibration targets. Biophysical and atmospheric 
characteristics of these targets are well instrumented 
on the ground. 


At one time approximately 250 images were processed 
by the EROS Data Center each day. Unfortunately, the 
ETM’ Scan Line Corrector (SLC) failed on May 31, 
2003, resulting in imagery with significant geometric 
error. The SLC compensates for the forward motion of 
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a. Band | (0.450 — 0.515 um). 


d. Band 4 (0.750 — 0.900 tm). 


Landsat 7 Enhanced Thematic Mapper Plus Imagery of San Diego, CA 
< > a™ 


g. Band 7 (2.08 — 2.35 um). 


h. Band 8 (0.52 — 0.90 um) panchromatic. 


i. Band 8 enlarged. 


FIGURE 2-21 Landsat 7 ETM* imagery of San Diego, CA, obtained on April 24, 2000. Bands 1 through 5 and 7 are 30 x 30 
m. Thermal infrared band 6 is 60 x 60 m. The panchromatic band 8 is 15 x 15 m (imagery courtesy of USGS and NASA). 


the satellite. Efforts to recover the SLC were not suc- 
cessful. Portions of ETM® scenes obtained after this 
date in SLC-off mode are usable after special process- 
ing (USGS Landsat 7, 2004). 


A USGS/NASA Landsat team refined ETM® gap-fill- 
ing techniques that merge data from multiple ETM* 
acquisitions. They also developed modifications to the 


Landsat-7 acquisition scheme to acquire two or more 
clear scenes as near in time as possible to facilitate this 
gap-filling process. These merged images resolve some 
of the missing data problems (NASA Landsat 7, 2014). 


Landsat 8 (LDCM-the Landsat Data Continuity Mis- 
sion) launched on February 11, 2013, uses linear array 
technology and is discussed in a subsequent section. 


Path 40, Row 37 
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Color-infrared color composite (RGB = Landsat ETM? bands 4, 3, 2). 


FIGURE 2-22 Color composite of Landsat 7 Enhanced Thematic Mapper Plus imagery of San Diego, CA, obtained on April 


24, 2000 (imagery courtesy of USGS and NASA). 


NOAA Multispectral Scanner Sensors 


NOAA operates two series of remote sensing satellites: 
the Geostationary Operational Environmental Satel- 
lites (GOES) and the Polar-orbiting Operational Envi- 
ronmental Satellites (POES). Both are currently based 
on multispectral scanner technology. The U.S. Nation- 
al Weather Service uses data from these sensors to fore- 


cast the weather. We often see GOES images of North 
and South America weather patterns on the daily news. 
The Advanced Very High Resolution Radiometer 
(AVHRR) was developed for meteorological purposes. 
However, global climate change research has focused 
attention on the use of AVHRR data to map vegeta- 
tion and sea-surface characteristics. 
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FIGURE 2-23 a) Geographic coverage of GOES East (75° W) and GOES West (135° W). b) Radiant flux from the terrain is re- 
flected off a scanning mirror (not shown) onto the primary and secondary mirrors. A dichroic beamsplitter separates the visi- 
ble light from the thermal infrared energy. Subsequent beamsplitters separate the thermal energy into specific bands. 


Geostationary Operational Environmental 
Satellite (GOES) 

The GOES system is operated by the National Envi- 
ronmental Satellite Data and Information Service 
(NESDIS) of NOAA. The system was developed by 
NESDIS in conjunction with NASA. GOES-N be- 
came operational on April 4, 2010, as GOES-13 East. 
Unfortunately, it failed on September 24, 2012, and 
GOES-O the backup satellite, was activated to become 
GOES-14 East. GOES P became operational on De- 
cember 6, 2011, and became GOES-15 West (NOAA 
GOES, 2014). 


GOES satellites are a mainstay of weather forecasting 
in the United States. They are the backbone of short- 
term forecasting or nowcasting. The real-time weather 
data gathered by GOES satellites, combined with data 
from Doppler radars and automated surface observing 
systems, greatly aid weather forecasters in providing 
warnings of thunderstorms, winter storms, flash 
floods, hurricanes, and other severe weather. These 
warnings help to save lives and preserve property 


The GOES system consists of several observing sub- 
systems: 


¢ GOES Imager (provides multispectral image data), 

¢ GOES Sounder (provides hourly 19-channel sound- 
ings), and 

¢ a data-collection system (DCS) that relays data 
from in situ sites at or near the Earth’s surface to 
other locations. 


The GOES spacecraft is a three-axis (x, y, z) stabilized 
design capable of continuously pointing the optical line 
of sight of the imaging and sounding radiometers to- 
ward the Earth. GOES are placed in geostationary or- 
bits approximately 35,790 km (22,240 statute miles) 
above the equator. The satellites remain at a stationary 
point above the equator and rotate at the same speed 
and direction as Earth. This enables the sensors to 
stare at a portion of the Earth from the geosynchro- 
nous orbit and thus more frequently obtain images of 
clouds, monitor the Earth’s surface temperature and 
water vapor characteristics, and sound the Earth’s at- 
mosphere for its vertical thermal and water vapor 
structures. 


The GOES East satellite is normally situated at 75°W 
longitude and GOES West is at 135°W longitude. The 
geographic coverage of GOES East and GOES West is 
summarized in Figure 2-23a. These sensors view most 
of the Earth from approximately 20°W to 165°E longi- 
tude. Poleward coverage is between approximately 77° 
N and S latitude. GOES East and West view the con- 
tiguous 48 states, South America, and major portions 
of the central and eastern Pacific Ocean and the central 
and western Atlantic Ocean. Pacific coverage includes 
the Hawaiian Islands and Gulf of Alaska, the latter 
known to weather forecasters as “the birthplace of 
North American weather systems.” 


GOES Imager The Imager is a five-channel multi- 
spectral scanner. The bandwidths and spatial resolu- 
tion are summarized in Table 2-5. By means of a two- 
axis gimballed mirror in conjunction with a 31.1 cm 
(12.2 in) diameter Cassegrain telescope, the Imager’s 
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TABLE 2-5 NOAA Geostationary Operational Environmental Satellite (GOES) Imager Instrument characteristics for GOES I-M 
(NOAA GOES, 2014). 


Typical Spectral Spatial 
GOES Resolution Resolution 
Bands (um) (km) Band Utility 
1 visible 0155 ='0:75 138 1 Clouds, pollution, haze detection, and identification of severe storms 
2 SWIR 3.80 — 4.00 4x4 Fog detection, discriminates between water, clouds, snow or ice clouds during 
daytime, detects fires and volcanoes, nighttime sea surface temperature (SST) 
3 Moisture 6.50 — 7.00 8x8 Estimation of mid- and upper-level water vapor, detects advection, and tracks 
mid-level atmospheric motion 
41R1 10.2 -11.2 4x4 Cloud-drift winds, severe storms, cloud-top heights, heavy rainfall 
5IR2 11.5 = 125 4x4 Identification of low-level water vapor, SST, and dust and volcanic ash 


multispectral channels can simultaneously sweep an 8 
km (5 statute mile) north-to-south swath along an east- 
to-west/west-to-east path, at a rate of 20° (optical) per 
second. This translates into being able to scan a 3,000 
by 3,000 km (1,864 by 1,864 miles) “box” centered over 
the United States in just 41 seconds. The actual scan- 
ning sequence takes place by sweeping in an east-west 
direction, stepping in the north-south direction, than 
sweeping back in a west-east direction, stepping north- 
south, sweeping east-west, and so on (NOAA GOES, 
2014). The telescope concentrates both the visible and 
thermal radiant flux from the terrain onto a secondary 
mirror (Figure 2-23b). Dichroic beamsplitters separate 
the incoming scene radiance and focus it onto 22 detec- 
tors (8 visible and 14 thermal). The visible energy pass- 
es through the initial beamsplitter and is focused onto 
8 silicon visible detector elements. Each of the 8 visible 
detectors has an IFOV of approximately 1 x 1 km at 
the satellite’s suborbital point on the Earth. 


All thermal infrared energy is deflected to the special- 
ized detectors in the radiative cooler. The thermal in- 
frared energy is further separated into the 3.9-, 6.75-, 
10.7-, and 12- um channels. Each of the four infrared 
channels has a separate set of detectors: four-element 
indium-antimonide (InSb) detectors for band 2; two- 
element mercury-cadmium-telluride (Hg:Cd:Te) detec- 
tors for band 3; and four-element mercury-cadmium- 
telluride (Hg:Cd:Te) detectors for both bands 4 and 5. 


The GOES channels have 10-bit radiometric precision. 
The primary utility of the visible band 1 (1 x 1 km) is 
in the daytime monitoring of thunderstorms, frontal 
systems, and tropical cyclones. Band 2 (4 x 4 km) re- 
sponds to both emitted terrestrial radiation and re- 
flected solar radiation. It is useful for identifying fog 
and discriminating between water and ice clouds, and 
between snow and clouds, and for identifying large or 
very intense fires. It can be used at night to track low- 


level clouds and monitor near-surface wind circulation. 
Band 3 (8 X 8 km) responds to mid- and upper-level 
water vapor and clouds. It is useful for identifying the 
jet stream, upper-level wind fields, and thunderstorms. 
Energy recorded by band 4 (4 x 4 km) is not absorbed 
to any significant degree by atmospheric gases. It is ide- 
al for measuring cloud-top heights, identifying cloud- 
top features, assessing the severity of some thunder- 
storms, and tracking clouds and frontal systems at 
night. Thermal band 5 (4 x 4 km) is similar to band 4 
except that this wavelength region has a unique sensi- 
tivity to low-level water vapor. GOES-8 East visible, 
thermal infrared, and water vapor images of Hurricane 
Bonnie on August 25, 1998, are shown in Figure 2- 
24a—e. GOES-12 visible imagery of Hurricane Katrina 
collected on August 29, 2005, and draped over a MO- 
DIS color composite is shown in Figure 2-25a. GOES- 
12 visible imagery obtained on August 28, 2005, is 
shown in Figure 2-25b. 


The Imager scans the continental United States every 
15 minutes; scans most of the hemisphere from near 
the North Pole to approximately 20° S latitude every 26 
minutes; and scans the entire hemisphere once every 3 
hours in “routine” scheduling mode. Optionally, spe- 
cial imaging schedules are available, which allow data 
collection at more rapid time intervals over reduced 
geographic areas. GOES-14 is providing | minute im- 
agery, which is giving scientists a sense of the type of 
imagery that GOES-R will provide after its launch in 
2015. As amazing as l-minute imagery appears, 
GOES-R will have the capability to provide 30-second 
imagery, which is 60 more frequent than GOES pro- 
vides routine imagery (NOAA GOES, 2014). 


GOES Sounder The GOES Sounder uses one visible 
and 18 infrared sounding channels to record data in a 
north-to-south swath across an east-to-west path. The 
Sounder and Imager both provide full Earth imagery, 
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Geostationary Operational Environmental Satellite (GOES) Imagery 


c. GOES-8 East visible August 25, 1998. 


e. GOES-8 East water vapor August 25, 1998. 


f. GOES-8 satellite. 


FIGURE 2-24 a-e) Examples of GOES-8 imagery obtained on August 25, 1998. f) The GOES-8 satellite (images courtesy of 


NOAA). 


sector imagery, and area scans of local regions. The 19 
bands yield the prime sounding products of vertical at- 
mospheric temperature profiles, vertical moisture pro- 
files, atmospheric layer mean temperature, layer mean 
moisture, total precipitable water, and the lifted index 


(a measure of stability). These products are used to 
augment data from the Imager to provide information 
on atmospheric temperature and moisture profiles, sur- 
face and cloud-top temperatures, and the distribution 
of atmospheric ozone. 
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GOES Imagery of Hurricane Katrina 


[Hurricane Katrina 


28 August 2005 
1445 GMT 


GOES-12 visibte 


a. GOES-12 visible band image draped over 
MODIS color composite on August 29, 2005. 


b. GOES-12 visible band imagery 
recorded on August 28, 2005. 


FIGURE 2-25 a) GOES-12 band 1 visible image of Hurricane Katrina on August 29, 2005, draped over a MODIS color com- 
posite (image courtesy of GOES Project NASA Goddard Space Flight Center). b) GOES-12 band 1 visible image of Hurricane 


Katrina on August 28, 2005 (image courtesy of NOAA). 


Advanced Very High Resolution Radiometer 
The Satellite Services Branch of the National Climatic 
Data Center, under the auspices of NESDIS, has estab- 
lished a digital archive of data collected from the 
NOAA Polar-orbiting Operational Environmental Sat- 
ellites (POES). This series of satellites commenced with 
TIROS-N (launched in October, 1978) and continued 
with NOAA-A (launched in March 1983, and renamed 
NOAA-8) to the current NOAA-19 launched in 2009. 
These Sun-synchronous polar-orbiting satellites carry 
the Advanced Very High Resolution Radiometer 
(AVHRR). Substantial progress has been made in us- 
ing AVHRR data for land-cover characterization and 
the mapping of daytime and nighttime clouds, snow, 
ice, and surface temperature. Unlike the Landsat TM 
and Landsat 7 ETM” sensor systems with nadir revisit 
cycles of 16 days, the AVHRR sensors acquire images 
of the entire Earth two times each day. This high fre- 
quency of coverage enhances the likelihood that cloud- 
free observations can be obtained for specific temporal 
windows and makes it possible to monitor change in 
land-cover conditions over short periods, such as a 
growing season. Moreover, the moderate resolution 
(1.1 x 1.1 km) of the AVHRR data makes it possible to 
collect, store, and process global datasets. For these 
reasons, NASA and NOAA initiated the AVHRR 
Pathfinder Program to create universally available 
global long-term remotely sensed datasets that can be 
used to study global climate change. 


The AVHRR satellites orbit at approximately 833 km 
above Earth at an inclination of 98.9° and continuous- 
ly record data in a swath 2,700 km wide at 1.1 x 1.1 km 


spatial resolution at nadir. Normally, two NOAA se- 
ries satellites are operational at one time (one odd, one 
even). The odd-numbered satellite typically crosses the 
equator at approximately 2:30 p.m. and 2:30 a.m., and 
the even-numbered satellite crosses the equator at 7:30 
p.m. and 7:30 a.m. local time. Each satellite orbits 
Earth 14.1 times daily (every 102 min) and acquires 
complete global coverage every 24 hours. 


The AVHRR is a cross-track scanning system. The 
scanning rate of the AVHRR is 360 scans per minute. 
A total of 2,048 samples (pixels) are obtained per 
channel per Earth scan, which spans an angle of 
+55.4° off-nadir. The IFOV of each band is approxi- 
mately 1.4 milliradians leading to a resolution at the 
satellite subpoint of 1.1 x 1.1 km. The band character- 
istics of the various AVHRR satellites are summarized 
in Table 2-6. The sensitivity of five of the NOAA-19 
bands is shown in Figure 2-26. 


Full-resolution AVHRR data obtained at 1.1 x 1.1 km 
are called local area coverage (LAC) data. They may be 
resampled to 1.1 x 4 km global area coverage (GAC) 
data. The GAC data contain only one out of three 
original AVHRR lines and the data volume and resolu- 
tion are further reduced by starting with the third sam- 
ple along the scan line, averaging the next four samples, 
and skipping the next sample. The sequence of average 
four, skip one is continued to the end of the scan line. 
Some studies use GAC data while others use the full- 
resolution LAC data. Please be aware that the AVHRR 
effective spatial resolution becomes very coarse near 
the edges of the scan swath. 
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TABLE 2-6 NOAA Advanced Very High Resolution Radiometer sensor system characteristics (NOAA AVHRR, 2014; USGS 


AVHRR, 2014). 


NOAA-6, 8, NOAA-7, NOAA-15, 16, 
10, 12 9,11, 13, 14 7a 187 19 
Spectral Spectral AVHRR/3 
Resolution Resolution Spectral 
Band (um)? (um)? Resolution 
(ym)? Band Utility 
1 Red 0.580 - 0.68 0.580 — 0.68 0.580 — 0.68 Daytime cloud, snow, ice, and vegetation map- 
ping; used to compute NDVI 
2 Near-IR 0.725 — 1.10 0.725 — 1.10 0.725 — 1.10 Land/water interface, snow, ice, and vegetation 
mapping; used to compute NDVI 
3 SWIR 3.55 — 3.93 3.55 — 3.93 3A: 1.58 - 1.64 Snow and ice detection 
3B: 3.55 — 3.93 Nighttime cloud mapping and sea surface temper- 
ature 
4 Thermal IR 10.50 - 11.50 10.30 - 11.30 10.30 - 11.30 Day/night cloud and surface-temperature map- 
ping 
5 Thermal IR None 11.50 - 12.50 11.50 — 12.50 Cloud and surface temperature, day and night 
cloud mapping; removal of atmospheric water va- 
por path radiance 
IFOV at nadir 1.1.x 1.1.km veh elt laa 1.1.x 1.1 km 
Swath width at 2,700 km 2,700 km 2,700 km 
nadir 


a. TIROS-N was launched on October 13, 1978; NOAA-6 on June 27, 1979; NOAA-7 on June 23, 1981; NOAA-8 on March 28, 1983; 
NOAA-9 on December 12, 1984; NOAA-10 on September 17, 1986; NOAA-11 on September 24, 1988; NOAA-12 on May 14, 1991; 
NOAA-13 on August 9, 1993; NOAA-14 on December 30, 1994; NOAA (K)-15 on May 13, 1998; NOAA (L)-16 on September 21, 
2000; NOAA (M)-17 on June 24, 2002; NOAA (N)-18 on May 20, 2005; NOAA (M)-19 on February 6, 2009. 


The AVHRR provides regional information on vegeta- 
tion condition and sea-surface temperature. Band 1 is 
approximately equivalent to Landsat TM band 3. Veg- 
etated land appears dark in this band of imagery due 
to chlorophyll absorption of red light. Band 2 is ap- 
proximately equivalent to TM band 4. Vegetation re- 
flects much of the near-infrared radiant flux while 
water absorbs much of the incident energy. The land— 
water interface is usually quite distinct. The three ther- 
mal bands provide information about Earth’s surface 
and water temperature. For example, Figure 2-27 is a 
sea-surface temperature map derived from NOAA-16 
AVHRR imagery obtained on October 16, 2003 
(Gasparovic, 2003). 


Scientists often compute a normalized difference vege- 
tation index (NDVI) from the AVHRR data using the 
visible (AVHRR) and near-infrared (AVHRR,>) bands 
to map the condition of vegetation on a regional and 
national level. It is a simple transformation based on 
the following ratio: 


Pnir~Preqd _ AVHRR, ~AVHRR, 


NDVI = nn 
AVHRR, + AVHRR, 


(2.4) 
Pnir si Pred 


The NDVI equation produces values in the range of 
~1.0 to 1.0, where increasing positive values indicate in- 
creasing green vegetation, and negative values indicate 
nonvegetated surfaces such as water, barren land, ice, 
and snow or clouds. To obtain the most precision, the 
NDVI is derived from calibrated, atmospherically cor- 
rected AVHRR channel | and 2 data in 16-bit preci- 
sion, prior to geometric registration and sampling. The 
final NDVI results from —1 to 1 are normally scaled 
from 0 to 200. Vegetation indices are discussed in 
Chapter 8. 


NDVI data obtained from multiple dates of AVHRR 
data can be composited to provide summary seasonal 
information. The n-day NDVI composite is produced 
by examining each NDVI value pixel by pixel for each 
observation during the compositing period to deter- 
mine the maximum value. The retention of the highest 
NDVI value reduces the number of cloud-contaminat- 
ed pixels. 


The NDVI and other vegetation indexes (refer to 
Chapter 8) have been used extensively with AVHRR 
data to monitor natural vegetation and crop condition, 
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FIGURE 2-26 Characteristics of NOAA-19 AVHRR bandwidths. 


identify deforestation in the tropics, and monitor areas 
undergoing desertification and drought. For example, 
the U.S. Geological Survey developed the Global Land 
Cover Characterization dataset based primarily on the 
unsupervised classification (refer to Chapter 9) of 1-km 
AVHRR 10-day NDVI composites. The AVHRR 
source imagery dates from April 1992 through March 
1993. Ancillary data sources include digital elevation 
data, ecoregions interpretation, and country- or re- 
gional-level vegetation and land-cover maps (USGS 
Global Landcover, 2014). 


NOAA Global Vegetation Index products based on 
AVHRR data are summarized as (NOAA GVI, 2014): 
first-generation (May 1982—April 1985); second-gener- 
ation (April 1985—present), and third-generation new 
products (April 1985—present). NOAA and NASA are 
currently developing a follow-on to AVHRR and MO- 
DIS for the operational NPOESS constellation. 


T 
NZe2 eee 


NOAA Suomi NPOESS Preparatory Project 
(NPP) 

The Suomi NPOESS Preparatory Project (NPP) was 
launched October 28, 2011. It orbits the Earth about 
14 times each day at 512 miles (824 km) above the 
Earth’s surface. The Suomi NPP is the first in a new 
generation of satellites intended to replace the NOAA 
Polar Operational Environmental Satellites (POES) 
that were launched from 1997 to 2011. NPP observes 
the Earth’s surface twice every 24-hour day, once in 
daylight and once at night.The NPP satellite is named 
after the late Verner E. Suomi of the University of Wis- 
consin. 


The Suomi NPP satellite has five distinct imaging sys- 
tems, including: 


* The Visible Infrared Imaging Radiometer Suite 
(VIIRS) collects data in 22 channels between 0.4 
and 12 um using whiskbroom scanner technology 
and 12-bit quantization. VIIRS has a swath width of 
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NOAA-16 Advanced Very High Resolution Radiometer (AVHRR) Imagery 
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Sea-surface temperature (SST) map derived from NOAA-16 AVHRR 
band 4 (10.3 - 11.3 um) imagery obtained on October 16, 2003. 


FIGUR 


2-27 Sea-surface temperature (SST) map derived from NOAA-16 AVHRR thermal infrared imagery (courtesy of 


NOAA and the Ocean Remote Sensing Program at Johns Hopkins University; Gasparovic, 2003). 


3,000 km. VIIRS has numerous bands that are very 
similar to those found on the Moderate Resolution 
Imaging Spectroradiometer (MODIS) operating on 
two NASA satellites, Terra and Aqua (NASA NPP, 
2011; NASA VURS, 2014). A comparison of the 
VIIRS and MODIS bands is found in Table 2-7. 
Characteristics of the VURS platform are shown in 
Figure 2-28ab. The data are used to observe active 
fires, vegetation, ocean color, the urban heat-island, 
sea-surface temperature, and other surface features. 
A Blue Marble composite image of the Earth cre- 
ated from VIIRS imagery collected on January 4, 
2012, is shown in Figure 2-28c. 

e Advanced Technology Microwave Sounder 
(ATMS), a microwave radiometer which will help 
create global moisture and temperature models; 

¢ Cross-Track Infrared Sounder (CrIS), a Michelson 
interferometer to monitor moisture and pressure; 

* Ozone Mapping and Profiler Suite (OMPS), a 
group of imaging spectrometers to measure ozone 
levels, especially near the poles; 


¢ Clouds and the Earth's Radiant Energy System 
(CERES), a radiometer to detect thermal radiation, 
including reflected solar radiation and thermal radi- 
ation emitted by the Earth. 


VIIRS in effect replaces three currently operating sen- 
sors: the Defense Meteorological Satellite Program 
(DMSP) Operational Line-scanning System (OLS), the 
NOAA Polar-orbiting Operational Environmental Sat- 
ellite (POES), Advanced Very High Resolution Radi- 
ometer (AVHRR), and the NASA Earth Observing 
System (EOS Terra and Aqua) Moderate-resolution 
Imaging Spectroradiometer (MODIS). 


The oceans cover more than two-thirds of the Earth’s 
surface and play an important role in the global cli- 
mate system. SeaWiFS was an advanced scanning sys- 
tem designed specifically for ocean monitoring. The 
SeaStar satellite (Orb View-2) carried the SeaWiFS into 
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a. NPP sensor systems. 


b. The NPP satellite in the 
clean room prior to launch. 


c. NASA composite of 
multiple VIIRS 
images collected on 
January 4, 2012 
into a “Blue Marble”. 


FIGURE 2-28 a) NPOESS Preparatory Project (NPP) satellite sensor systems. b) The NPP satellite in a clean room prior to 
launch. c) Blue Marble rendition of multiple NPP VIIRS images obtained on January 4, 2012 (NASA/NOAA/GSFC/Suomi 


NPP/VIIRS/Norman Kuring). 


orbit using a Pegasus rocket on August 1, 1997 (NA- 
SA/Orbimage SeaWiFS, 2014). The final orbit was 705 
km above the Earth. The equatorial crossing time was 
12 p.m. 


SeaWiFS 

SeaWiFS built on all that was learned about ocean col- 
or remote sensing using the Nimbus-7 satellite Coastal 
Zone Color Scanner (CZCS) launched in 1978. CZCS 
ceased operation in 1986. The SeaWiFS instrument 
consisted of an optical scanner with a 58.3° total field 
of view. Incoming scene radiation was collected by a 


telescope and reflected onto the rotating half-angle 
mirror. The radiation was then relayed to dichroic 
beamsplitters that separate the radiation into eight 
wavelength intervals (Table 2-8). SeaWiFS had a spa- 
tial resolution of 1.13 x 1.13 km (at nadir) over a swath 
of 2,800 km. It had a revisit time of 1 day. 


SeaWiFS recorded energy in eight spectral bands with 
very narrow wavelength ranges (Table 2-8) tailored for 
the detection and monitoring of very specific ocean 
phenomena, including ocean primary production and 
phytoplankton processes, ocean influences on climate 
processes (heat storage and aerosol formation), and the 
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TABLE 2-7 Characteristics of the NPOESS Preparatory Project (NPP) Visible Infrared Imaging Radiometer Suite (VIIRS) bands 
and their relationship with MODIS bands (Gleason et al., 2010). 


VIIRS Spectral Resolution Spatial MODIS Spectral Resolution Spatial 
Band (um) Resolution (m) Bands (um) Resolution (km) 
DNB 0.5-0.9 
M1 0.402 — 0.422 750 8 0.405 — 0.420 1,000 
M2 0.436 — 0.454 750 9 0.438 — 0.448 1,000 
M3 0.478 — 0.498 750 3 or 10 0.459 —- 0.479 500 
0.483 - 0.493 1,000 
M4 0.545 — 0.565 750 4 or 12 0.545 — 0.565 500 
0.546 — 0.556 1,000 
11 0.600 — 0.680 375 1 0.620 — 0.670 250 
M5 0.662 — 0.682 750 13 or 14 0.662 — 0.672 1,000 
0.673 — 0.683 1,000 
M6 0.739 — 0.754 750 iS 0.743 - 0.753 1,000 
12 0.846 — 0.885 S75 2 0.841 — 0.876 250 
M7 0.846 — 0.885 750 16 or 2 0.862 — 0.877 1,000 
0.841 — 0.876 250 
M8 1.230 - 1.250 750 5 Same 500 
M9 1.371 - 1.386 750 26 1.360 - 1.390 1,000 
13 1.580 - 1.640 375 6 1.628 - 1.652 500 
M10 1.580 - 1.640 750 6 1.628 - 1.652 500 
M11 222012275) 750 7 2.105 — 2.115 500 
14 3.550 — 3.930 375 20 3.660 — 3.840 1,000 
M12 3.550 — 3.930 750 20 same 1,000 
M13 3.973 — 4.128 750 21 or 22 3929 73.989 1,000 
3.929 — 3.989 1,000 
M14 8.400 — 8.700 750 29 same 1,000 
M15 10.263 — 11.263 750 31 10.780 — 11.280 1,000 
15 10.500 — 12.400 375 31 or 32 10.780 — 11.280 1,000 
11.770 — 12.270 
M16 11.538 — 12.488 750 32 11.770 — 12.270 1,000 


cycles of carbon, sulfur, and nitrogen. In particular, | wavelength absorption), at 490 nm (to increase sensi- 
SeaWiFS had specially designed bands centered at 412 __ tivity to chlorophyll concentration), and in the 765 and 
nm (to identify yellow substances through their blue 
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TABLE 2-8 Characteristics of the Sea-viewing Wide Field-of-view Sensor. 


SeaWiFS Band Center Band Width 
Band (nm) (nm) Band Utility 

1 412 402 — 422 Identify yellow substances 
2 443 433 - 453 Chlorophyll concentration 
3 490 480 — 500 Increased sensitivity to chlorophyll concentration 
4 510 500 — 520 Chlorophyll concentration 
5 555) 545 — 565 Gelbstoffe (yellow substance) 
6 670 660 — 680 Chlorophyll concentration 
7 765 745 -— 785 Surface vegetation, land-water interface, atmospheric correction 
8 865 845 - 885 Surface vegetation, land-water interface, atmospheric correction 


865 nm near-infrared (to assist in the removal of atmo- 
spheric attenuation). 


SeaWiFS observations helped scientists understand the 
dynamics of ocean and coastal currents, the physics of 
mixing, and the relationships between ocean physics 
and large-scale patterns of productivity. The data filled 
the gaps in ocean biological observations between 
those of the test-bed CZCS and MODIS. SeaWiFS 
ceased operation on February 15, 2011. Information 
about obtaining historical SeaWiFS data can be found 
at http://oceancolor.gsfc.nasa.gov/Sea WiFS/. 


Multispectral Imaging Using 
Linear Arrays 


Linear array sensor systems use diodes or charge-cou- 
pled-devices to record the reflected or emitted radiance 
from the terrain. Linear array sensors are often called 
“pushbroom” sensors because, like a single line of bris- 
tles in a broom, the linear array stares constantly at the 
ground while the aerial or satellite platform moves for- 
ward (Figures 2-9d and 2-29a). The result is a more ac- 
curate measurement of the reflected radiant flux 
because 1) there is no moving mirror, and 2) the linear 
array detectors are able to dwell longer on a specific 
portion of the terrain. 


NASA Earth Observing-1 (EO-1) 
Advanced Land Imager (ALI) 


The NASA Earth Observing-1 (EO-1) satellite was 
launched on November 21, 2000, as part of a one-year 


technology validation/demonstration mission. The 
mission operated three advanced technology verifica- 
tion land imaging instruments. They were the first 
Earth-observing instruments to be flown under NA- 
SA’s New Millennium Program. The three instruments 
are the Advanced Land Imager (ALI), the Hyperion 
hyperspectral imager, and the Linear Etalon Imaging 
Spectrometer Array (LEISA) Atmospheric Corrector 
(LAC). These instruments were supposed to enable fu- 
ture Landsat and Earth observing missions to more ac- 
curately classify and map land utilization globally. EO- 
1 was launched on November 21, 2001, into a polar or- 
bit with an equatorial crossing time of 10:03 a.m. (de- 
scending node), an altitude of 705 km, an inclination 
of 98.2°, and an orbital period of 98 minutes. An agree- 
ment was reached between NASA and the USGS to al- 
low continuation of the EO-1 Program as an Extended 
Mission. The EO-1 Extended Mission is chartered to 
collect and distribute ALI multispectral and Hyperion 
hyperspectral products in response to Data Acquisi- 
tion Requests (DARs). Under the Extended Mission 
provisions, image data acquired by EO-1 are archived 
and distributed by the USGS Center for Earth Re- 
sources Observation and Science (EROS) and placed in 
the public domain (http://eo1-usgs.gov). 


The Advanced Land Imager (ALI) 


The ALI pushbroom instrument on EO-1 was used to 
validate and demonstrate technology for the Landsat 
Data Continuity Mission (LDCM). The ALI flies in 
formation with Landsat 7 ETM” (Digenis, 2005). The 
ALT has nine multispectral bands and a panchromatic 
band, three more than ETM”, but does not have the 
thermal band (Mendenhall et al., 2012). It has in- 
creased sensitivity by a factor varying from four to ten 
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Coupled-Devices (CCDs) 


Linear Array 
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FIGURE 2-29 a) Enlarged view of a 2,048 element charge-coupled-device (CCD) 
linear array. b) Enlarged view of a 3,456 x 2,048 area array CCD. 


EO-1 Advanced Land Imager (ALI) and Hyperion Hyperspectral Sensor 


a. ALI image of Oahu, HI (RGB = bands 3,2,1). 


Earth Observing 
System 


b. ALI image of Pearl Harbor, HI, 
pansharpened (RGB = bands 3,2,1). 


<7.5 km—» 


c. Hyperion image 
of San Francisco, CA 
(RGB = bands 30,21,15). 


FIGURE 2-30 a) Mosaicked image of Oahu, HI, created using four 37 km swaths of EO-1 Advanced Land Imager (ALI) data. 
b) Enlargement of pan-sharpened ALI image of Pearl Harbor, HI. c) EO-1 Hyperion hyperspectral image of San Francisco, CA, 
with a swath width of 7.5 km (images courtesy of NASA Goddard Space Flight Center; Jensen et al., 2012). 


depending upon the band. The spatial resolution of the 
multispectral bands is the same as that of ETM™ (30 x 
30 m) but the panchromatic band has improved spatial 
resolution (10 <x 10 m versus 15 x 15 m). The wave- 
length coverage and ground sampling distance (GSD) 
are summarized in Table 2-9. Six of the nine multispec- 
tral bands are the same as those of the ETM* on 
Landsat 7, enabling direct comparison. The ALI has a 
swath width of 37 km. ALI images of Oahu, HI, are 
shown in Figure 2-30ab. 


EO-1 specifications are summarized in Table 2-4. It 
contains a linear array Advanced Land Imager (ALI) 
with 10 bands from 0.4 to 2.35 um at 30 x 30 m spatial 
resolution. The Hyperion hyperspectral sensor records 
data in 220 bands from 0.4 to 2.4 um at 30 x 30 m spa- 
tial resolution. The Linear Etalon Imaging Spectrome- 
ter Array (LEISA) Atmospheric Corrector is a 256- 
band hyperspectral instrument sensitive to the region 
from 0.9 to 1.6 um at 250 x 250 m spatial resolution. It 
is designed to correct for water-vapor variations in the 


TABLE 2-9 Landsat 8 characteristics (Markham, 2011). 
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Landsat 8 Landsat 8 
Operational Land Imager (OLI) Thermal Infrared Sensor (TIRS) 
Band Spectral Spatial Band Spectral Spatial 
Resolution Resolution Resolution Resolution 
(um) (m) at Nadir (um) (m) at Nadir 
1 Ultra-blue for 0.433 - 0.453 30 x 30 10 Thermal OB = 11138} 100 x 100 
coastal/aerosol 
2 Blue 0.450 — 0.515 30 x 30 11 Thermal 11.5-12.5 100 x 100 
3 Green 0.525 — 0.600 30 x 30 Sensor 
Technology Pushbroom 
4 Red 0.630 — 0.680 30 x 30 Swath Width 185 km 
5 Near-infrared 0.845 - 0.885 30 x 30 Data Rate 400 WRS-2 scenes per day 
6 SWIR-1 1.56 — 1.66 30 x 30 Revisit 16 days 
7 SWIR-2 2.1-2.3 30 x 30 Orbit and 705 km, Sun-synchronous 
Inclination Inclination = 98.2° 
Equatorial crossing 10:00 a.m. 
8 Panchromatic 0.52 — 0.90 532 15 Quantization 12-bits (dramatic improvement over ETM*) 
9 Cirrus 1.36 - 1.39 30 x 30 Signal to Noise 1-2 orders of magnitude improvement 


atmosphere. All three of the EO-1 land imaging instru- 
ments view all or sub-segments of the Landsat 7 swath. 


NASA Landsat 8 (LDCM—Landsat 
Data Continuity Mission) 


Landsat 8 (LDCM—the Landsat Data Continuity 
Mission) is a partnership between NASA and the 
USGS. Landsat 8 was launched on February 11, 2013. 
The goal of Landsat 8 is to: 1) collect and archive me- 
dium resolution (circa 15- to 30-m spatial resolution) 
multispectral image data affording seasonal coverage 
of the global landmasses for a period of no less than 5 
years, 2) ensure that Landsat 8 data are sufficiently 
consistent with data from the earlier Landsat missions 
in terms of acquisition geometry, calibration, coverage 
characteristics, spectral characteristics, output product 
quality, and data availability to permit studies of land- 
cover and general land-use change over time, and 3) 
distribute Landsat 8 data products to the public via the 
Internet on a nondiscriminatory basis and at a price no 
greater than the incremental cost of fulfilling a user re- 
quest (USGS, 2010). 


Orbital Land Imager 


Landsat 8 is planned as a 5-year mission but will in- 
clude enough fuel for 10 years of operation. The multi- 


spectral remote sensing instrument is the Operational 
Land Imager (OLI). The OLI collects land-surface 
data with a spatial and spectral resolution consistent 
with historical Landsat TM and ETM” data previously 
discussed at 15 x 15 m (panchromatic) and 30 x 30 m 
(multispectral VNIR/SWIR) (Figure 2-17). It also has 
two additional spectral channels: an “ultra-blue” band 
for coastal and aerosol studies and a band for cirrus 
cloud detection. The OLI uses long detector arrays, 
with approximately 7,000 detectors per spectral band, 
aligned across its focal plane to view across the swath. 
The 15 x 15 m panchromatic band has more than 
13,000 detectors. Silicon PIN (SiPIN) detectors that 
collect the data for the visible and near-infrared spec- 
tral bands (Bands | to 4 and 8) while mercury—cadmi- 
um-telluride (Hg:Cd:Te) detectors are used for the 
shortwave infrared bands (Bands 6, 7, and 9). This 
“pushbroom” design results in a more sensitive instru- 
ment providing improved land surface information 
with fewer moving parts (Irons et al., 2012). 


The thermal infrared sensor (TIRS) collects data in 
two long wavelength bands that are co-registered with 
the OLI data. About 400 scenes per day will be collect- 
ed and terrain-corrected to a geographic projection. 
OLI and TIRS data are distributed as a combined 
product. In 2007, a plan was put forth for a National 
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b. SPOT 5 image of 
San Francisco, CA, 
obtained on July 9, 2005. 


FIGURE 2-31 Chronological launch history of the SPOT satellites (updated from Jensen et al., 2012; SPOT 5 image courtesy 


of Airbus Defense and Space). 


Land Imaging Program that includes calling for opera- 
tional moderate-resolution land imaging into the fu- 
ture (USGS, 2010). Landsat 9 is in the preliminary 
planning stages for launch in approximately 2018. 


SPOT Sensor Systems 


The first SPOT satellite was launched on February 21, 
1986 (Figure 2-31). It was developed by the French 
Centre National d’Etudes Spatiales (CNES) in cooper- 
ation with Belgium and Sweden. It had a spatial reso- 
lution of 10 x 10 m (panchromatic mode) and 20 x 20 
m (multispectral mode) and provided several other in- 
novations in remote sensor system design (Table 2-10). 
SPOT satellites 2 and 3 with identical payloads were 
launched on January 22, 1990, and September 25, 
1993, respectively. SPOT 4 was launched on March 24, 
1998, with a Vegetation sensor (1 X 1 km) and a short- 
wavelength infrared band. SPOT 5 was launched on 
May 3, 2002, with visible, near-infrared, and shortwave 
infrared (SWIR) bands at 10 x 10 m and a panchro- 
matic band at 2.5 x 2.5 m. SPOT 6 was launched on 
September 9, 2012. SPOT was the first commercially- 
available satellite system based on linear array technol- 
ogy and the first pointable sensor. 


Since 1986, SPOT Earth observation satellites have 
been a consistent, dependable source of high-resolu- 
tion Earth resource information. While many countries 
have seen their Earth resource monitoring sensor sys- 
tems come and go depending primarily upon politics, 
one could always count on SPOT Image, Inc. to pro- 
vide quality imagery. Unfortunately, the imagery has 
always been relatively expensive, usually more than 
$2,000 per panchromatic or multispectral scene, al- 
though it has been reduced in recent years. The ex- 
pense can be substantial if both panchromatic and 
multispectral imagery of the study area are required. 


SPOT 1, 2, and 3 

These satellites are all identical and consist of two 
parts: 1) the SPOT bus, which is a standard multipur- 
pose platform, and 2) the sensor system instruments 
(Figure 2-32ab) consisting of two high-resolution visi- 
ble (HRV) sensors and a package comprising two data 
recorders and a telemetry transmitter. The satellites 
operate in a Sun-synchronous, near-polar orbit (incli- 
nation of 98.7°) at an altitude of 822 km. The satellites 
pass overhead at the same solar time; the local clock 
time varies with latitude. 


TABLE 2-10 SPOT 1, 2, and 3 High-Resolution Visible (HRV), SPOT 4 and 5 High-Resolution Visible and Infrared (HRVIR), and 
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SPOT 4 and 5 Vegetation sensor system characteristics. 


SPOT 1, 2, 3 HRV and 4 HRVIR SPOT 5 HRVIR SPOT 4 and 5 Vegetation 
Band Spectral Spatial Band Spectral Spatial Band Spectral Spatial 
Resolution Resolution Resolution Resolution Resolution Resolution 
(um) (m) at Nadir (um) (m) at (um) (km) at 
Nadir Nadir 
1 0.50 - 0.59 20 x 20 1 0.50 - 0.59 10 x 10 1 eee > 
2 0.61 - 0.68 20 x 20 2 0.61 - 0.68 10 x 10 2 Vole bees Tels 
3 0.79 - 0.89 20 x 20 3 0.79 - 0.89 10 x 10 3 UES) > 
Pan 0.51 - 0.73 10 x 10 Pan 0.48 - 0.71 DIS 58 5 
Pan (4) 0.61 - 0.68 10 x 10 
SWIR (4) | 1.58-1.75 20 x 20 SWIR 1.58- 1.75 20 x 20 SWIR BE HS ee 
Sensor Linear array pushbroom Linear array pushbroom Linear array pushbroom 
Swath 60 km + 50.5° 60 km + 27° 2 250i kip Ore 
Rate 25 Mb/s 50 Mb/s 50 Mb/s 
Revisit 26 days 26 days 1 day 
Orbit 822 km, Sun-synchronous 822 km, Sun-synchronous 822 km, Sun-synchronous 
Inclination = 98.7° Inclination = 98.7° Inclination = 98.7° 
Equatorial crossing 10:30 a.m. Equatorial crossing 10:30 a.m. Equatorial crossing 10:30 a.m. 


The HRV sensors operate in two modes in the visible 
and reflective-infrared portions of the spectrum. The 
first is a panchromatic mode corresponding to obser- 
vation over a broad spectral band (similar to a typical 
black-and-white photograph). The second is a multi- 
spectral (color) mode corresponding to observation in 
three relatively narrow spectral bands (Table 2-10). 
Thus, the spectral resolution of SPOTs 1 through 3 is 
not as good as that of the Landsat Thematic Mapper. 
The ground spatial resolution, however, is 10 x 10 m 
for the panchromatic band and 20 x 20 m for the three 
multispectral bands when the instruments are viewing 
at nadir, directly below the satellite. 


Reflected energy from the terrain enters the HRV via a 
plane mirror and is then projected onto two CCD ar- 
rays. Each CCD array consists of 6,000 detectors ar- 
ranged linearly. An electron microscope view of some 
of the individual detectors in the linear array is shown 
in Figure 2-33ab. This linear array pushbroom sensor 
images a complete line of the ground scene in the cross- 
track direction in one look as the sensor system pro- 
gresses downtrack (refer to Figure 2-5d). This capabili- 
ty breaks tradition with the Landsat MSS, Landsat 


TM, and Landsat 7 ETM™ sensors because no me- 
chanical scanning takes place. A linear array sensor is 
superior because there is no mirror that must scan 
back and forth to collect data (mirror-scan velocity is a 
serious issue) and this allows the detector to literally 
“stare” at the ground for a longer time, obtaining a 
more accurate record of the spectral radiant flux exit- 
ing the terrain. The SPOT satellites pioneered this lin- 
ear array pushbroom technology in commercial Earth 
resource remote sensing as early as 1986. 


When looking directly at the terrain beneath the sensor 
system, the two HRV instruments can be pointed to 
cover adjacent fields, each with a 60-km swath width 
(Figure 2-32c). In this configuration the total swath 
width is 117 km and the two fields overlap by 3 km. It 
is also possible to selectively point the mirror to off- 
Nadir viewing angles through commands from the 
ground station. In this configuration it is possible to 
observe any region of interest within a 950-km wide 
strip centered on the satellite ground track (1.e., the ob- 
served region may not be centered on the ground 
track) (Figure 2-34a). The width of the swath actually 
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FIGURE 2-32 a-c) The SPOT satellites consist of the SPOT bus, which is a multipurpose platform, and the sensor system 
payload. Two identical high-resolution visible (HRV) sensors are found on SPOTs 1, 2, and 3 and two identical high-resolution 
visible infrared (HRVIR) sensors on SPOT 4. Radiant energy from the terrain enters the HRV or HRVIR via a plane mirror and is 
then projected onto two CCD arrays. Each CCD array consists of 6,000 detectors arranged linearly. This results in a spatial 
resolution of 10 x 10 or 20 x 20 m, depending on the mode in which the sensor is being used. The swath width at nadir is 60 
km. The SPOT HRV and HRVIR sensors may also be pointed off-Nadir to collect data. SPOT 4 carries a Vegetation sensor with 
1.15 x 1.15 km spatial resolution and 2,250-km swath width (adapted from SPOT Image, Inc.). 


observed varies between 60 km for nadir viewing and 
80 km for extreme off-nadir viewing. 


If the HRV instruments were capable only of Nadir 
viewing, the revisit frequency for any given region of 
the world would be 26 days. This interval is often unac- 
ceptable for the observation of phenomena evolving on 
time scales ranging from several days to a few weeks, 
especially where cloud cover hinders the acquisition of 
usable data. During the 26-day period separating two 
successive SPOT satellite passes over a given point on 
Earth and taking into account the steering capability 
of the instruments, the point in question could be ob- 
served on seven different passes if it were on the equa- 


tor and 11 times if at a latitude of 45° (Figure 2-34b). A 
given region can be revisited on dates separated alter- 
natively by 1 to 4 (or occasionally 5) days. 


The SPOT sensors can also acquire cross-track stereo- 
scopic pairs of images of a geographic region (Figure 
2-34c). Two observations can be made on successive 
days such that the two images are acquired at angles on 
either side of the vertical. In such cases, the ratio be- 
tween the observation base (distance between the two 
satellite positions) and the height (satellite altitude) is 
approximately 0.75 at the equator and 0.50 at a latitude 
of 45°. SPOT data with these base-to-height ratios may 
be used for topographic mapping. 
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FIGURE 2-33 a) Scanning electron microscope images of the front surface of a CCD linear array like that used in the SPOT 
HRV sensor systems. Approximately 58 CCD detectors are visible, with rows of readout registers on both sides. b) Seven de- 
tectors of a CCD linear array are shown at higher magnification. This document is taken from the CNES Web site www.cnes.fr. 


Protected information. All rights reserved ' CNES (2014). 


SPOT 10 x 10 m panchromatic data are of such high 
geometric fidelity that they can be photointerpreted 
like a typical aerial photograph in many instances. For 
this reason, SPOT panchromatic data are often regis- 
tered to topographic base maps and used as orthopho- 
tomaps. Such image maps are useful in GIS databases 
because they contain more accurate planimetric infor- 
mation (e.g., new roads, subdivisions) than out-of-date 
7.5-min topographic maps. The improved spatial reso- 
lution available is demonstrated in Figure 2-35, which 
presents a TM band 3 image and a SPOT panchromat- 
ic image of Charleston, SC. 


SPOT sensors collect data over a relatively small 60 x 
60 km (3,600 km”) area compared with Landsat MSS 
and TM image areas of 170 x 185 km (31,450 km?) 
(Figure 2-36). It takes 8.74 SPOT images to cover the 
same area as a single Landsat TM or MSS scene. This 
may be a limiting factor for regional studies. However. 
SPOT does allow imagery to be purchased by the km? 
(e.g., for a watershed or school district) or by the linear 
km (e.g., along a highway). 


SPOTs 4 and 5 

SPOT Image, Inc., launched SPOT 4 on March 24, 
1998, and SPOT 5 on May 3, 2002. Their characteris- 
tics are summarized in Table 2-10. The viewing angle 
can be adjusted to +27° off-nadir. SPOTs 4 and 5 have 
several notable features of significant value for Earth 
resource remote sensing, including: 


¢ the addition of a short-wavelength infrared (SWIR) 
band (1.58 — 1.75 um) for vegetation and soil mois- 
ture applications at 20 x 20 m; 

* an independent sensor called Vegetation for small- 
scale vegetation, global change, and oceanographic 
studies; 


¢ SPOT 4 has onboard registration of the spectral 
bands, achieved by replacing the original HRV pan- 
chromatic sensor (0.51—0.73 um) with band 2 
(0.61 — 0.68 um) operating in both 10- and 20-m res- 
olution mode; and 


« SPOT 5 panchromatic bands (0.48 —0.7 um) can 
collect 2.5 x 2.5 m imagery. 


Because the SPOT 4 and 5 HRV sensors are sensitive 
to SWIR energy, they are referred to as HRVIR 1 and 
HRVIR 2. 


The SPOT 4 and 5 Vegetation sensor is independent of 
the HRVIR sensors. It is a multispectral electronic 
scanning radiometer operating at optical wavelengths 
with a separate objective lens and sensor for each of 
the four spectral bands (blue = 0.43 — 0.47 um used pri- 
marily for atmospheric correction; red = 0.61 — 0.68 
um; near-infrared = 0.78 — 0.89 um; and SWIR = 1.58 
— 1.75 um). Each sensor is a 1,728 CCD linear array lo- 
cated in the focal plane of the corresponding objective 
lens. The spectral resolution of the individual bands 
are summarized in Table 2-10. The Vegetation sensor 
has a spatial resolution of 1.15 x 1.15 km. The objec- 
tive lenses offer a field of view of +50.5°, which trans- 
lates into a 2,250-km swath width. The Vegetation 
sensor has several important characteristics: 


¢ multidate radiometric calibration accuracy better 
than 3% and absolute calibration accuracy better 
than 5% is superior to the AVHRR, making it more 
useful for repeatable global and regional vegetation 
surveys; 


* pixel size is uniform across the entire swath width, 
with geometric precision better than 0.3 pixels and 


80 


INTRODUCTORY DIGITAL IMAGE PROCESSING 


SPOT Off-Nadir 
Viewing 


collection 


Off-nadir data 
collection 
= 


Orbital track 
at nadir 


a. 


SPOT Stereoscopic Viewing Capabilities 


Pass on day D Pass on day D + 1 


Stereoscopic 
model 


interband multidate registration better than 0.3 km 
because of the pushbroom technology; 


10:30 a.m. equatorial crossing 
AVHRR’s 2:30 p.m. crossing time; 


time versus 


a short-wavelength infrared band for improved veg- 
etation mapping; 


it is straightforward to relate the HRVIR 10 x 10m 


One pass on days: D+ 10 D+5 D 


SPOT Off-Nadir Revisit Capabilities 


FIGURE 2-34 a) The SPOT HRV instruments are pointable and 
can be used to view areas that are not directly below the satellite 
(i.e., off-nadir). This is useful for collecting information in the event 
of a disaster, when the satellite track is not optimum for collecting 
stereoscopic imagery. b) During the 26-day period separating two 
successive SPOT satellite overpasses, a point on Earth could be 
observed on different passes if it is at the equator and on 11 occa- 
sions if at a latitude of 45°. A given region can be revisited on 
dates separated alternatively by 1, 4, and occasionally 5 days. c) 
Two observations can be made on successive days such that the 
two images are acquired at angles on either side of the vertical, 
resulting in stereoscopic imagery. Such imagery can be used to 
produce topographic and planimetric maps (adapted from SPOT 
Image, Inc.). 


or 20 X 20 m data nested within the Vegetation 2,250 
x 2,250 km swath width data; and 


individual images can be obtained or data can be 
summarized over a 24-hr period (called a daily syn- 
thesis), or daily synthesis data can be compiled into 
n-day syntheses. 


SPOT 4 and 5 are still operational (SPOT Image, 2014; 
SPOT Payload, 2014). 
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a. Landsat Thematic Mapper 
Band 3 (30 x 30 m) on February 3, 1994. 


b. SPOT HRV Panchromatic 
Band (10 x 10 m) on January 10, 1996. 


FIGURE 2-35 Comparison of the detail in a) 30 x 30 m Landsat TM band 3 data (courtesy of NASA), and b) SPOT 10 x 10 m 
panchromatic data of Charleston, SC [SPOT image courtesy of CNES (www.cnes.fr). Protected information. All rights 


reserved ' CNES (2014)]. 


Landsat Thematic Mapper (TM) 
and Multispectral Scanner (MSS) 
image area 


FIGURE 2-36 Geographic coverage of the SPOT HRV and 
Landsat Multispectral Scanner and Thematic Mapper re- 
mote sensing systems. 


More than 20 million SPOT images have been collect- 
ed since 1986. 


SPOT 6 and 7 

SPOT 6 was launched on September 9, 2012. SPOT 7 
was launched on June 30, 2014 (Figure 2-31) (Astrium 
SPOT 6 and 7, 2014). They are to be placed in phase in 
the same orbit and are pointable. They will have a 1.5 x 
1.5 m panchromatic band and four 6 < 6 m multispec- 
tral bands with a 60 km swath width. Both cross-track 
and along-track stereoscopic data collection is possi- 
ble. Astrium GEO-Information Services tasks the 
French SPOT satellite family, Germany’s TerraSAR-X, 


and Taiwan’s FORMOSAT-2 directly, enabling rapid 
and reliable data acquisition of any place on the Earth 
each day (Astrium, 2014). 


Pleiades 


The SPOT 6 optical Earth observation satellite com- 
bines the advantages of a relatively wide imaging swath 
and 1.5 x 1.5 m spatial resolution. However, the CNES 
recognized the need for civilian and military applica- 
tions that require higher spatial resolution and the col- 
lection of data through clouds and at night. Therefore, 
a joint French-Italian Optical and Radar Federated 
Earth Observation (ORFEO) space program was initi- 
ated. The Pleiades optical system is supervised by the 
French CNES and the radar imaging system is super- 
vised by the Italian space agency (ISA) (CNES Pleia- 
des, 2014). 


Pleiades 1A and 1B 


There are two Pleiades satellites. Pleiades 1A was suc- 
cessfully launched on December 17, 2011, into a Sun- 
synchronous circular orbit at 694 km AGL. Pleiades 
1B was launched in 2013 (Figure 2-31). Each has a life 
span of 5 years. Pleiades satellites record panchromatic 
imagery (0.48 — 0.83 um) using linear array technology 
(Figure 2-5d) at 0.5 x 0.5 m spatial resolution at Nadir 
(Astrium, 2014). They also record four bands of multi- 
spectral data (blue [0.43 — 0.55 um], green [0.49 — 0.61 
um], red [0.60 — 0.72 um], and near-infrared [0.75 — 
0.95 um]) at 2 x 2 m spatial resolution with a swath 
width of 20 km at Nadir (Astrium Pleiades, 2014). 
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French CNES Pleiades HR-1 Satellite 


a. Pleiades 
HR-1 satellite. 


c. Pleiades HR-1 pansharpened color-composite image of Washington, DC. 
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37 a) Pleiades HR-1 panchromatic image of San Francisco, CA. b) Pansharpened image of Washington, DC (imag- 


es courtesy of ASTRIUM Geo-Information Services, Inc.; www.astrium-geo.com/). 


Merged panchromatic/multispectral images are avail- 
able as orthophoto products. Pleiades is pointable and 
can obtain stereoscopic images suitable for photo- 
grammetric analysis to extract detailed planimetric and 
three-dimensional information. Pleiades has a revisit 
capability of two days with one satellite and <24 hours 
with two satellites. Examples of Pleiades imagery of 
San Francisco, CA, and Washington, DC, are shown 
in Figure 2-37. 


Ine F 


The Indian Space Research Organization (ISRO) satel- 
lite system is one of the largest and most diverse con- 
stellations of remote sensing satellites in operation in 
the world today (www.isro.org). It began with the 
launch of the Indian Remote Sensing (IRS) satellites: 
IRS-1A on March 17, 1988, IRS-1B on August 29, 
1991, IRS-1C on December 28,1995, and IRS-1D on 
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Chronological Launch History of the 
Indian Space Research Organization (ISRO) Constellation of Satellites 
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FIGURE 2-38 The chronological launch history of the Indian Space Research Organization (ISRO) optical satellites and 
their range of spatial resolution (information courtesy of ISRO (www.IRSO.org) and ESA CEOS (2014); Jensen et al., 2012). 
b) The CartoSat-2 LISS-III multispectral digital camera (National Remote Sensing Centre (NRSC)/Indian Space Research 
Organization). 


was every 22 days at the equator (11-day repeat cov- 
erage with two satellites), with an orbital inclination 
of 99.5°. The swath width was 146 to 148 km. 


IRS-1A, -1B, -1C, and -1D * The IRS-1C and IRS-1D satellites carried three sen- 
¢ The IRS-1A and IRS-1B satellites acquired data sors: the LISS-I multispectral sensor, a panchro- 
with Linear Imaging Self-scanning Sensors (LISS-I matic sensor, and a Wide Field Sensor (WiFS). The 
and LISS-IJ) at spatial resolutions of 72.5 x 72.5 m LISS-III had four bands with the green, red, and 


September 29, 1997 (Table 2-11). The chronological 
launch history is shown in Figure 2-38. 


and 36.25 x 36.25 m, respectively. The data were 
collected in four spectral bands that were nearly 
identical to the Landsat TM visible and near-infra- 
red bands. The satellite’s altitude was 904 km, the 
orbit was Sun-synchronous, and repeat coverage 


near-infrared bands at 23.5 x 23.5 m spatial resolu- 
tion and the short-wavelength infrared band at 70.5 
x 70.5 m spatial resolution. The swath width was 
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TABLE 2-11 Indian CartoSat and ResourceSat characteristics (ISRO ResourceSat-1, 2014; ISRO ResourceSat-2, 2014). 


CartoSat ResourceSat 
Spectral Spatial Spectral Spatial 
Resolution Resolution Resolution Resolution 
(um) (m) at Nadir Band (um) (m) at Nadir 
CartoSat-1 0.50 - 0.75 Pops) 88 P25) ResourceSat-1 
(2 cameras) ResourceSat-2 
CartoSat-2 0.50 - 0.75 <a LISS-IV 2 0.52 -— 0.59 5.8 x 5.8 
CartoSat-2A 0.50 - 0.75 <—cl 3 0.62 — 0.68 5.8 x 5.8 
4 0.77 — 0.86 Bis) 86 {8} 
LISS-III 2 0.52 —0.59 PRS 86 D5) 
3} 0.62 — 0.68 PRIS 3% DBS) 
4 0.77 — 0.86 ZR 3% BBS) 
5 1.55 =1.70 ZRIS 3% BBS) 
AWiFS 2 0.52 —0.59 56 x 56 
3 0.62 — 0.68 56 x 56 
4 0.77 — 0.86 56 x 56 
5 1.55 = 1.70 56 x 56 
Sensor Linear array pushbroom Linear array pushbroom 
Swath width CartoSat-1 = 30 km; ResourceSat-1 (LISS-III = 141 km; LISS-IV = 23 km multispectral mode 
CartoSat-2 = 9.6 km; and 70 km mono mode; AWiFS = 740 km) 
CartoSat-2A = 9.6 km; ResourceSat-2 (LISS-II] = 141 km; LISS-IV = 70 km; AWiFS = 740 km) 
CartoSat-2B= 9.6 km 


141 km for bands 2, 3, and 4 and 148 km for band 5. 
Repeat coverage was every 24 days at the equator. 


¢ The panchromatic sensor had a spatial resolution of 
approximately 5.2 x 5.2 m, stereoscopic imaging 
capability, and a 70 km swath width with repeat cov- 
erage every 24 days at the equator and a revisit time 
of 5 days with +26° off-nadir viewing. 


¢ The Wide Field Sensor had 188 x 188 m spatial res- 
olution. The WiFS had two bands comparable to 
NOAA’s AVHRR satellite (0.62 — 0.68 um and 0.77 
— 0.86 um) with a swath width of 692 km. Repeat 
coverage was 5 days at the equator. 


CartoSat 


The ISRO has launched numerous high-resolution re- 
mote sensing systems that are especially designed for 


cartographic purposes, including: CartoSat-1, Carto- 
Sat-2, CartoSat-2A, and CartoSat-2B. The chronologi- 
cal launch history is shown in Figure 2-38. 


CartoSat-1 was launched on May 5, 2005. It carries 
two panchromatic cameras that have a spatial resolu- 
tion of 2.5 x 2.5 m with a 30 km swath width (Table 2- 
11). The cameras are mounted so that near simultane- 
ous imagery of the same geographic area from two dif- 
ferent angles is possible yielding stereoscopic imagery. 
CartoSat-1A launched August 1, 2014, records pan- 
chromatic (1.25 x 1.25 m; 500 — 750 nm; 60 km swath 
width), multispectral VNIR (2.5 x 2.5 m; 60 km swath 
width), and hyperspectral imagery (30 x 30 m; 60 km 
swath width; VNIR 750 — 1300 nm; SWIR 30 x 30 m; 
1,300 — 3,000 nm). CartoSat-1B may be launched in 
2017 with similar instruments (ESA CEOS, 2014). 
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ResourceSat-2 Imagery of Flooding on the Brahmaputra River in India in 2012 


aa - 7 Me 


a. Pre-flood ResourceSat-2 (LISS-III) image 
collected in February, 2012. 


b. Post-flood ResourceSat-2 (LISS-III) image 
collected on June 29, 2012. 


FIGURE 2-39 Indian ResourceSat-2 pre- and post-flood imagery of parts of the Bibrugarh, Sibsagar, and Dhemaji Districts in 


India (images courtesy of Indian Space Research Organization). 


CartoSat-2 was launched on January 10, 2007. It car- 
ries a single panchromatic camera with sensitivity from 
500 — 750 nm that collects data at < 1 x 1 m spatial res- 
olution with a 9.6 km swath width. The camera can be 
steered up to +45° along-track as well as across-track 
to obtain stereoscopic imagery. CartoSat-2A was 
launched on April 28, 2008. Its specifications are al- 
most identical to CartoSat-2. CartoSat-2B was 
launched on July 12, 2010, into a circular polar Sun- 
synchronous orbit with 14 orbits per day and a revisit 
cycle of 4 days. The sensor has a single panchromatic 
band (500 — 750 nm) at a spatial resolution of < 1 x 1 
m (ISRO CartoSat-2B, 2014). It has a swath width of 
9.6 km and is steerable up to +26° along-track as well 
as across-track to obtain stereoscopic imagery and 
achieve a four-day revisit capability. The sensor is de- 
signed for detailed urban and infrastructure planning 
and development, transportation system planning, and 
preparation of large-scale cartographic maps (ISRO 
CartoSat-2B, 2014). According to the European Space 
Agency (ESA), CartoSat-2C was launched July 31, 
2014 with a 1 to 2 m spatial resolution, 10 km swath 
width, and four bands in the VNIR (400 — 1,300 nm) 
and CartoSat-2D may be launched in 2016 with char- 
acteristics similar to CartoSat-2C (ESA CEOS, 2014). 
CartoSat-3 to be launched in 2017 will have a panchro- 
matic camera that acquires imagery at 25 x 25 cm spa- 
tial resolution in the region from 500 — 750 nm. It will 
have a swath width of 15 km. CartoSat-3A with similar 
sensor characteristics is to be launched in 2018 (ESA 
CEOS, 2014). 


ResourceSat 


In addition to CartoSat, ISRO has also launched sev- 
eral Earth resource remote sensing satellites, including: 


ResourceSat-1 and ResourceSat-2. ResourceSat-1 was 
launched on October 17, 2003, to replace and improve 
the remote sensing capability of the IRS-1C and IRS- 
1D Earth resource satellites (Figure 2-38). It carried 
three cameras including the high-resolution Linear Im- 
aging Self Scanner (LISS-IV) with three bands at 5.8 x 
5.8 m spatial resolution, steerable +26° across-track 
viewing to achieve five-day revisit capability; a LISS- 
III sensor operating in three spectral bands in the 
VNIR and one in the Short Wave Infrared (SWIR) 
band with 23.5 x 23.5 spatial resolution; and a coarse- 
resolution Advance Wide Field Sensor (AWiFS) oper- 
ating in three spectral bands in VNIR and one band in 
SWIR at 56 X 56 m. It has a revisit capability of 5 days. 
ResourceSat-2 was launched on April 20, 2011. It car- 
ries the three cameras that are similar to those of Re- 
sourceSat-1, including the LISS-IV, LISS-III (Figure 
2-38), and the AWiFS. Important changes in Resource- 
Sat-2 compared to ResourceSat-1 include: enhance- 
ment of LISS-IV multispectral swath from 23 km to 70 
km and improved radiometric accuracy from 7-bits to 
10-bits for LISS-II and LISS-IV and 10-bits to 12-bits 
for AWIFS. A follow-on satellite ResourceSat-2A is 
scheduled for launch in 2016 (ESA CEOS, 2014). Ex- 
amples of pre- and post-flood ResourceSat-2 imagery 
of an area in India are shown in Figure 2-39. 


Korean Aerospace Research Institute 
(KARI) KOMPSATs 


The Korea Aerospace Research Institute (KARI) su- 
pervises Korea’s satellite programs (GlobalSecuri- 
ty.org). The KOMPSAT (Korea Multi-Purpose 
Satellite) program was initiated in 1995 (Figure 2-40). 
Its objective is the development of a national space 
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Chronological Launch History of the Korean KOMPSAT Satellites 


2000 


9596 [97/58 [BB] 00 [or Joz oa] + | os fl o7 Jos [os [10] 1s fi 


% 3 j 


12.21.99 


+% 


PAN=Im 


Multispectral = 4m 


aed 


i 


— 
‘ 


| 


2010 2020 
oOo 
05.18.12 


KOMPSAT-3 


PAN = 0.8 m 
Multispectral = 4 m 


| 
08.22.13 


RADAR= | to 20m 


2014-2015 


PAN = 0.8m 
Multispectral = 4m 
Thermal = 5.5 m 


KOMPSAT-2 panchromatic | x 1 m image of San Francisco, CA. 


FIGURE 2-40 The chronological launch history of the Korean Aerospace Research Institute (KARI) satellites and their spatial 
resolutions. A KOMPSAT-2 panchromatic 1 x 1 m image of San Francisco International Airport obtained in 2006 is shown 
(image ' Korean Aerospace Research Institute (KARI), 2006; information courtesy of KARI and SPOT Image, Inc.; Jensen 


et al., 2012). 


segment in Earth observation along with an efficient 
infrastructure and ground segment to provide services 
to remote sensing users. 


KOMPSAT-1 (Arirang-1) was launched on December 
21, 1999, and retired on January 1, 2008. The payload 
included a CCD pushbroom Electro-Optical Camera 
(EOC) with a 6.6 < 6.6 m spatial resolution in the re- 
gion from 510 — 730 nm. It had a 28 revisit cycle and a 
swath width of 17 km (Figure 2-40). 


KOMPSAT-2 (Arirang-2) was launched on July 27, 
2006, and orbits the Earth 14 times a day. It hasa 1 x 1 
m panchromatic sensor (500 — 900 nm), a 4 x 4 m four- 
band VNIR multispectral camera, and a 15 km swath 
width (Figure 2-40). KARI developed the KOMPSAT- 
2 program in collaboration with EADS Astrium, Inc., 
to acquire VHR imagery for South Korea’s mapping 


requirements, urban planning, and hazard manage- 
ment. A KOMPSAT-2 image of San Francisco Inter- 
national Airport is shown in Figure 2-40. 


KOMPSAT-3 was launched on May 18, 2012. It has an 
Advanced Electronic Image Scanning System (AEISS) 
with a 0.8 x 0.8 m panchromatic band and a 4 x 4m 
VNIR multispectral sensor system (Figure 2-40). The 
swath width is 15 km. 


KOMPSAT-3A may be launched in 2014-2015. It will 
have panchromatic and multispectral remote sensing 
capability similar to KOMPSAT-3 except for a thermal 
infrared band in the 3 — 6 um region at a spatial resolu- 
tion of 5.5 x 5.5 m. It will have a 15 km swath width 
(ESA CEOS, 2014). There is no KOMPSAT-4 pro- 
gram. The Sino-Korean word for the number four, 
“sa” is a homonym of the Chinese character for death. 


Proposed Launch History of 
the ESA Sentinel Satellites 
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FIGURE 2-41 Proposed launch history of the Global Moni- 
toring for Environment and Security (GMES) program Euro- 
pean Space Agency Sentinel 1-3 satellites (information 
courtesy of ESA CEOS, 2014; updated from Jensen et al., 
2012). 


KOMPSAT-5 provides high-resolution Synthetic Ap- 
erture Radar (SAR) images at | x 1 m spatial resolu- 
tion, standard mode SAR images of 3 X 3 resolution, 
and wide swath mode SAR images of 20 x 20 m resolu- 
tion with an incidence angle of 45° using the COSI 
(COrea SAR Instrument) payload. KOMPSAT-5 was 
launched August 22, 2013. It has an X-band (12.5 — 8 
GHz) Synthetic Aperture Radar (SAR) with a 100 km 
swath width and a 28-day revisit cycle (ESA CEOS, 
2014; eoPortal KOMPSAT-5, 2014) (Figure 2-40). 
KOMPSAT-6 will be a SAR and is scheduled for 
launch in 2019. 


Astrium, Inc. Sentinel-2 


The Global Monitoring for Environment and Security 
(GMES) program is headed by the European Commis- 
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sion (EC) in partnership with the European Space 
Agency (ESA) and the European Environment Agency 
(EEA). The ambitious Earth observation program is to 
provide accurate, timely and easily accessible informa- 
tion to improve the management of the environment, 
understand and mitigate the effects of climate change 
and ensure civil security. Sentinel-la is a C-band radar 
satellite launched on April 3, 2014 (Figure 2-41). Senti- 
nel-2 records high-resolution optical multispectral im- 
agery for land applications. Sentinel-3 will provide data 
for ocean and land applications. Sentinel-4 and Senti- 
nel-5 will provide data for atmospheric composition 
monitoring from geostationary and polar orbits, re- 
spectively (ESA GMES Sentinel-2, 2014; ESA Senti- 
nel-2, 2014). Sentinel-2 services fall into six main 
categories: services for land management, services for 
the marine environment, services relating to the atmo- 
sphere, services to aid emergency response, services as- 
sociated with security, and services relating to climate 
change. The mission will adhere to the Sentinel Data 
Policy, which establishes full and open access to data 
acquired by all of the Sentinels. 


A pair of Sentinel-2(ab) satellites will deliver high-reso- 
lution optical images globally, providing enhanced 
continuity of SPOT and Landsat-type data. The first 
satellite is scheduled to launch in 2015 and the next 
within 18 months. Astrium GmbH is the prime con- 
tractor and has responsibility for the design, develop- 
ment, and the integration of the Sentinel-2 satellite 
(Figure 2-41). Sentinel-2 will carry a Multi-Spectral In- 
strument (MSI) with a total of 13 bands: 4 VNIR 
bands (400 — 750 nm) at 10 m, 6 SWIR bands (1,300 — 
3,000 nm) at 20 m and 3 bands at 60 m spatial dedicat- 
ed to atmospheric corrections and cloud screening. It 
has a swath width of 290 km (ESA CEOS, 2014). The 
mission orbits at a mean altitude of approximately 800 
km and, with the pair of satellites in operation, will 
have a revisit time of 5 days at the equator (under 
cloud-free conditions) and 2—3 days at mid-latitudes. 
The increased swath width along with the short revisit 
time allow rapid changes to be monitored, such as veg- 
etation during the growing season. Data from Sentinel- 
2 will benefit services associated with land manage- 
ment by European and national institutes, the agricul- 
tural industry and forestry, as well as disaster control 
and humanitarian relief operations. 


Imagery for the generation of high-level operational 
products, such as land-cover maps, land-change detec- 
tion maps and geophysical variables that use, for exam- 
ple, leaf-area-index, leaf chlorophyll content and leaf 
water content will be provided. Images of floods, volca- 
nic eruptions and landslides will also be acquired by 
Sentinel-2. In essence, Sentinel-2 combines a large 
swath, frequent revisit, and systematic acquisition of 
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TABLE 2-12 NASA Advanced Spaceborne Thermal Emission and Reflection Radiometer characteristics. 


Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) 
Band VNIR Band SWIR Band TIR 
Spectral Spectral Spectral 
Resolution Resolution Resolution 
(um) (um) (um) 
1 (nadir) 0.52 — 0.60 1.600 — 1.700 10 8.125 — 8.475 
2 (nadir) 0.63 - 0.69 2.145 — 2.185 11 8.475 — 8.825 
3 (nadir) 0.76 — 0.86 2.185 — 2.225 12 8.925 — 9.275 
3 (backward) 0.76 — 0.86 2.235 — 2.285 13 10.25 — 10.95 
2.295 — 2.365 14 10.95 — 11.65 
2.360 — 2.430 
Technology Pushbroom Pushbroom Whiskbroom 
(detector) Si PtSi:Si Hg:Cd:Te 
Spatial 
resolution (m) (155 Se 115) 30 x 30 90 x 90 
Swath width 60 km 60 km 60 km 
Quantization 8 bits 8 bits 12 bits 


all land surfaces at high spatial resolution and with a 
large number of spectral bands, all of which makes a 
unique mission to serve GMES. 


Advanced Spaceborne Thermal 
Emission and Reflection Radiometer 
(ASTER) 


The ASTER is a joint effort between NASA and Ja- 
pan’s Ministry of International Trade and Industry. 
ASTER collects information on surface temperature, 
emissivity, reflectance, and elevation (NASA ASTER, 
2014a). It is the only high-to-moderate-spatial-resolu- 
tion instrument on the Terra satellite. It is used in con- 
junction with MODIS, MISR, and CERES sensors 
that monitor the Earth at moderate to coarse spatial 
resolutions. ASTER serves as a zoom lens for the other 
Terra instruments and is particularly important for 
change detection and calibration/validation studies. 


ASTER collects data in 14 channels from the visible 
through the thermal-infrared regions of the electro- 
magnetic spectrum. The sensor system consists of three 
separate instrument subsystems. The individual band- 
widths and subsystem characteristics are summarized 
in Table 2-12. 


The VNIR detector subsystem operates in three spec- 
tral bands in the visible and near-infrared wavelength 
region with a spatial resolution of 15 x 15 m. It con- 
sists of two telescopes—one nadir-looking with a 
three-spectral-band CCD detector and another back- 
ward-looking with a single-band CCD detector. The 
backward-looking telescope provides a second view of 
the study area in band 3 for stereoscopic observations. 
Across-track pointing to 24° off-nadir is accomplished 
by rotating the entire telescope assembly. ASTER opti- 
cal VNIR imagery of Waikiki, HI, is shown in Figure 
2-42. 


The SWIR subsystem collects data in six spectral 
bands in the 1.6 — 2.43 um region through a Nadir- 
pointing telescope that provides 30 x 30 m spatial reso- 
lution. Cross-track pointing (+ 8.55°) is accomplished 
by a pointing mirror. 


The TIR subsystem operates in five bands in the ther- 
mal-infrared region using a single nadir-looking tele- 
scope with a spatial resolution of 90 x 90 m. Unlike the 
other subsystems, it has a whiskbroom scanning sys- 
tem. Each band uses 10 detectors with optical band- 
pass filters over each detector element. The scanning 
mirror functions both for scanning and cross-track 


Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Optical Imagery 
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Diamond Head 


a. Visible near-infrared (VNIR) image of Waikiki, HI, 


(15 x 15 m spatial resolution; RGB = bands 2,3,1). 
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b. 3x magnification. 


FIGURE 2-42 Terra ASTER imagery of Waikiki and Diamond Head on the island of Oahu, Hawaii (courtesy of NASA/GSFC/ 
MITI/ERSADC/JAROS and U.S./Japan ASTER Science Team and the California Institute of Technology). 


ASTER Thermal-infrared Images of a Lava Flow on Hawaii 


a. 


Cc. 


FIGURE 2-43 A sequence of Terra ASTER nighttime 90 x 90 m thermal infrared band 14 (10.95 — 11.65 ym) images of Pu’u 
O’'o lava flows entering the sea at Kamokuna on the southeast side of the Island of Hawaii. The images were obtained on a) 
May 22, 2000, b) June 30, 2000, and c) August 1, 2000 (courtesy of NASA/GSFC/MITI/ERSADC/JAROS and U.S./Japan AS- 


TER Science Team and the California Institute of Technology). 


pointing (+ 8.55°). During scanning, the mirror rotates 
90° from nadir to view an internal blackbody. Multi- 
date nighttime 90 x 90 m thermal infrared band 14 
(10.95 — 11.65 um) images of Pu’u O’o lava flows enter- 
ing the sea at Kamokuna on the southeast side of the 
Island of Hawaii are shown in Figure 2-43. 


Multiangle Imaging 
Spectroradiometer (MISR) 


The Multiangle Imaging Spectroradiometer was built 
by NASA’s Jet Propulsion Laboratory and is one of the 
five Terra satellite instruments. MISR measures the 
Earth’s brightness in four spectral bands, at each of 
nine look angles spread out in the forward and aft di- 
rections along the flight line. Spatial samples are ac- 


quired every 275 m. Over a period of 7 minutes, a 360- 
km wide swath of Earth comes into view at all nine an- 
gles (NASA MISR, 2014). 


An illustration of the nine look angles is shown in Fig- 
ure 2-44. The digital pushbroom sensors image the 
Earth at 26.1°, 45.6°, 60°, and 70.5° forward and aft of 
the local vertical (Nadir 0°). Note that the fore and aft 
camera angles are arranged symmetrically about nadir. 
In general, large viewing angles provide enhanced sen- 
sitivity to atmospheric aerosol effects and to cloud re- 
flectance effects, whereas more modest angles are 
required for land-surface viewing. 


Each MISR camera sees instantaneously a single row 
of pixels in a pushbroom format. It records data in 
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Multiangle Imaging Spectroradiometer (MISR) 
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FIGURE 2-44 Artist's rendition of the Multiangle Imaging 
Spectroradiometer (MISR) on EOS Terra. MISR uses linear 
array technology to acquire imagery of the terrain in four 
bands at nine angles: at Nadir (0°) and at 26.1°, 45.6°, 60°, 
and 70.5° forward and aft of nadir (adapted from NASA Jet 
Propulsion Laboratory). 


four bands: blue, green, red, and near-infrared. The in- 
dividual band wavelengths are summarized in Figure 
2-44. Each camera has four independent linear CCD 
arrays (one per filter), with 1,504 active pixels per lin- 
ear array. 


The nadir-viewing camera (labeled An in Figure 2-44) 
provides imagery that is less distorted by surface topo- 
graphic effects than that of any other MISR camera. It 
also is the least affected by atmospheric scattering. It 
provides 1) useful reference for navigating within all the 
MISR imagery, and 2) a base image to compare with 
images acquired at different angles of view. Such com- 
parisons provide important “bidirectional reflectance 
distribution function, BRDF” information introduced 
in Chapter 1. The nadir-viewing camera also offers an 
opportunity to compare observations with other nadir- 
viewing sensors such as Landsat TM and ETM”. The 
nadir-viewing camera also facilitates calibration. 


The fore and aft 26.1° view angle cameras (Af and Aa) 
provide useful stereoscopic information that can be of 
benefit for measuring topographic elevation and cloud 
heights. The fore and aft 45.6° view angle cameras (Bf 
and Ba) are positioned to be especially sensitive to at- 
mospheric aerosol properties. The fore and aft 60° view 
angle cameras (Cf and Ca) provide observations look- 
ing through the atmosphere with twice the amount of 
air than the vertical view. This provides unique infor- 
mation about the hemispherical albedo of land surfac- 
es. The fore and aft 70.5° view angle cameras (Df and 
Da) provide the maximum sensitivity to off-nadir ef- 
fects. The scientific community is interested in obtain- 
ing quantitative information about clouds and the 
Earth’s surface from as many angles as possible. 


Geo Eye, Inc. (formerly Space 
Imaging, Inc.), IKONOS-2, GeoEye-1, 
GeoEye-2 


In 1994, the U.S. government made a decision to allow 
civil commercial companies to market high spatial res- 
olution remote sensor data (approximately | x 1 to 4 x 
4m). This resulted in the creation of a number of com- 
mercial consortiums that had the capital necessary to 
create, launch, and market high spatial resolution digi- 
tal remote sensor data. The most notable companies 
are Space Imaging, Inc., Orbimage, Inc., and Earth- 
Watch, Inc. These companies targeted the geographic 
information system (GIS) and cartographic mapping 
markets traditionally serviced by the aerial photogram- 
metric industries. The commercial remote sensing 
firms hoped to have an impact in markets as diverse as 
agriculture, natural resource management, local and 
regional government, transportation, emergency re- 
sponse, mapping, and eventually an array of average 
consumer applications such as Google Earth. Warfare 
during the early twenty-first century also created a sig- 
nificant military demand for commercial high spatial 
resolution imagery. 


All commercial vendors offer an Internet online order- 
ing service and a suite of products that can be tailored 
to user requirements, including the creation of digital 
elevation models from the remote sensor data. The 
commercial remote sensing companies typically price 
the imagery according to the type of product ordered 
and the amount of geographic coverage desired (km2). 
The sensors used by these companies are based primar- 
ily on linear array CCD technology. 


IKONOS-1 and -2 

Space Imaging, Inc. launched IKONOS-1 on April 27, 
1999. Unfortunately, the IXKONOS-1 satellite never 
achieved orbit. Space Imaging, Inc. successfully 
launched the IXONOS-2 satellite on September 24, 
1999 (Figure 2-45). The IKONOS-2 sensor system has 
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FIGURE 2-45 The chronological launch history of the IKONOS and GeoEye satellites and their range of spatial resolution. 
IKONOS-2 and GeoEye-1 images of Minot, ND, are shown (images courtesy of DigitalGlobe, Inc.; Jensen et al., 2012). 


a 1 X 1 m panchromatic band (Figure 2-45) and four 
multispectral visible and near-infrared bands at 4 x 4 
m spatial resolution. Sensor characteristics are summa- 
rized in Table 2-13. IKONOS-2 is in a Sun-synchro- 
nous 681-km orbit, with a descending equatorial 
crossing time of between 10 and 11 a.m. It has both 
cross-track and along-track viewing instruments, 
which enable flexible data acquisition and frequent re- 
visit capability: <3 days at 1 x 1 m spatial resolution 
(for look angles < 26°) and 1.5 days at 4 x 4 m spatial 
resolution. The nominal swath width is 11 km. Data 
are quantized to 11-bits. 


GeoEye-1 and -2 

GeoEye, Inc. was established in 2006 as a result of the 
merger of Orbital Imaging, Inc. and Space Imaging, 
Inc. It launched GeoEye-1 on September 6, 2008. It 
has a 41 < 41 cm panchromatic band and four multi- 
spectral bands at 1.65 x 1.65 m spatial resolution (Ta- 
ble 2-13). IKONOS-2 and GeoEye-1| images are shown 
in Figure 2-45. GeoEye, Inc. was merged into Digital- 
Globe, Inc. on January 29, 2013. GeoEye-2 (to be 
known as WorldView-4) is to be launched in 2016. It 


will have 0.34 x 0.34 m spatial resolution, four multi- 
spectral bands, revisit frequency of < 3 days, 14 km 
swath width, and positional accuracy of < 5 m CE90 
(GeoEye, 2014). Until recently, the Federal govern- 
ment only permitted distribution of imagery at a reso- 
lution of better 50 <x 50 cm outside of the USS. 
government and its cooperators (USGS, 2014). 


The Federal government Enhanced View program was 
a $7.3 billion, 10-year contract for commercial satellite 
imagery from two publicly traded U.S. companies: 
GeoEye, Inc. and DigitalGlobe, Inc. These companies 
invested >$1 billion in new satellite remote sensing in- 
frastructure based on this government program, and 
the majority of their revenues come from government 
contracts. 


EarthWatch/DigitalGlobe, Inc., 
QuickBird, WorldView-1, WorldView- 
2, WorldView-3 


EarthWatch, Inc. was one of the pioneers in commer- 
cial high spatial resolution satellite remote sensing. 
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TABLE 2-13 Sensor characteristics of satellites operated by GeoEye, Inc., which has merged with DigitalGlobe, Inc. 


IKONOS-2 (1999) GeoEye-1 (2008) GeoEye-2 (WorldView-4) 
Band Spectral Spatial Band Spectral Spatial Band Spectral Spatial 
Resolution Resolution Resolution Resolution Resolution Resolution 

(nm) (m) at Nadir (nm) (m) at Nadir (nm) (m) at Nadir 
1 445-516 ADs OD 1 450-510 1.65 x 1.65 1 450-510 1.36 x 1.36 
2 506 — 595 Bese. 2 510 — 580 1.65 x 1.65 2 510-580 1.36 x 1.36 
3 632 — 698 AD BD 3 655 — 690 1.65 x 1.65 3 655 — 690 1.36 x 1.36 
4 757 — 853 Bese 4 780 — 920 1.65 x 1.65 4 780 - 920 1.36 x 1.36 
Pan 526 - 929 0.82 x 0.82 | Pan 450 — 800 0.41 x 0.41 Pan 450 — 510 0.34 x 0.34 
Sensor Linear array pushbroom Linear array pushbroom Linear array pushbroom 

QuickBird EROS A and EROS B 


EarthWatch, Inc., launched EarlyBird in 1996 with a 3 
x 3 m panchromatic band and three visible to near-in- 
frared (VNIR) bands at 15 x 15 m spatial resolution. 
Unfortunately, EarthWatch, Inc. lost contact with the 
satellite shortly after launch. DigitalGlobe, Inc. 
launched QuickBird on October 18, 2001, with a 65 x 
65 cm panchromatic band and four 2.62 x 2.62 m mul- 
tispectral bands (Figure 2-46; Table 2-14). The Quick- 
Bird orbit was recently raised to 482 km AGL which 
results in a slight decrease in spatial resolution (Digi- 
talGlobe QuickBird, 2014; USGS, 2013). 


WorldView-1, -2, and -3 

DigitalGlobe, Inc. launched WorldView-1 on Septem- 
ber 18, 2007, with a 0.5 x 0.5 m panchromatic band 
(DigitalGlobe WorldView-1, 2014). WorldView-2 was 
launched on October 8, 2009, with a 46 x 46 cm pan- 
chromatic band and eight multispectral bands at 1.85 
x 1.85 m spatial resolution (Table 2-14) (DigitalGlobe 
WorldView-2, 2014; USGS, 2013). DigitalGlobe 
launched WorldView-3 on August 13, 2014. It has one 
panchromatic band (450 — 800 nm) at 31 x 31 cm spa- 
tial resolution, eight multispectral bands at 1.24 x 1.24 
m spatial resolution, and 8 SWIR bands with 3.7 x 3.7 
m resolution (DigitalGlobe World View-3, 2014). 


ImageSat International, Inc., EROS A 
and EROS B 


ImageSat International, Inc. is a commercial provider 
of high-resolution, satellite Earth-imagery collected by 
its Earth Remote Observation Satellites (EROS) (Im- 
ageSat EROS, 2014). 


ImageSat, Inc. successfully launched its first satellite, 
EROS A, on December 5, 2000. In so doing, ImageSat 
became the second company in the world to successful- 
ly deploy a non-government-owned high-resolution 
imaging satellite. ImageSat, Inc. successfully launched 
its second satellite, EROS B, on April 25, 2006 (Figure 
2-47). 


The EROS satellites acquire high-resolution panchro- 
matic (500 — 900 nm) image data using pushbroom lin- 
ear array technology at 10-bit radiometric resolution. 
The satellites are deployed in a Sun-synchronous near- 
polar orbit at an altitude of approximately 500 km. 
EROS A and B satellites orbit the earth approximately 
15 times each day delivering imagery data in real-time 
to ground receiving stations worldwide. EROS A and B 
sensors can be pointed and stabilized to image custom- 
er specified sites at nadir (perpendicular to the surface) 
or at oblique angles up to 45°. Oblique viewing enables 
the satellite to view any site on the Earth as often as 
two to three times per week. 


EROS A is equipped with a camera whose focal plane 
of CCD (Charge-Coupled Device) detectors produces 
a standard image spatial resolution of 1.9 x 1.9 m with 
a swath of 14 km at nadir. The EROS B satellite has a 
larger panchromatic band camera of CCD/TDI type 
(Charge-Coupled Device/Time Delay Integration) with 
a spatial resolution of 0.70 x 0.70 m, a larger onboard 
recorder, improved pointing accuracy and a faster data 
communication link. The expected lifespan of both sat- 
ellites is 14 years. Recently, ImageSat, Inc. and 
RapidEye, Inc. reached an agreement whereby EROS 
panchromatic data can be fused with RapidEye multi- 
spectral data (ImageSat EROS, 2014). 


CHAPTER 2. REMOTE SENSING DATA COLLECTION 93 


Chronological Launch History of the 
DigitalGlobe Satellites 


2000 


2010 2020 


3798] 99] 00 B02 03 [0% | 0s [os J os 
10.18.01 


QuickBird 
a) 4 
i 


PAN = 0.65 m 
Multispectral = 2.62 m 


a 


ems 


PAN = 0.5m 


10.08.09 


« .. e 
rs 2 ’ WorldView-2 
oe P/E 


x 


PAN = 0.46 m 
Multispectral = 1.85 m 


08.13.14 


+ 


af 


PAN = 0.31 m 
Multispectral = 1.24 m 
SWIR = 3.7m 


WorldView-2 image of San Diego, CA, obtained on 
May 3, 2010, at 0.5 x 0.5 m spatial resolution. 


FIGURE 2-46 The chronological launch history of the QuickBird, WorldView-1, -2, and -3. A WorldView-2 image of San 
Diego, CA, is shown (images courtesy of DigitalGlobe, Inc.; updated from Jensen et al., 2012). 


RapidEye, Inc. 


The RapidEye constellation of five identical Earth ob- 
servation satellites was launched on August 29, 2008, 
(using a refurbished ICBM missile) and became opera- 
tional in February, 2009 (Figure 2-48). The constella- 
tion acquires over 4 million km? of Earth imagery each 
day and has amassed more than 2.5 billion km? of mul- 
tispectral imagery. 


RapidEye 

The RapidEye satellite was launched on August 29, 
2008. The RapidEye multispectral pushbroom sensor 
has five bands at 5 X 5 m spatial resolution, a swath 
width of 77 km, with a daily revisit time. RapidEye’s 
satellites are the first commercial satellites to offer the 
Red-edge band (690 — 730 nm), which measures vari- 
ances in vegetation allowing for species separation and 
monitoring vegetation health (Figure 2-48). The Red- 
edge is of value for discriminating between urban and 
vegetation land cover (Dykstra, 2012; RapidEye, 
2014). RapidEye level 1b is radiometrically corrected 
only. The RapidEye Ortho Product has radiometric, 
sensor, and geometric corrections applied to the data 


and is rectified using DTED Level | SRTM DEM or 
better. With appropriate ground control it can meet an 
accuracy of 6 m (1 sigma or 12.7 m CE90) meeting 
1:25,000 NMAS - standards (RapidEye, 2014). 
RapidEye, Inc., imagery is distributed in the US. 
through Blackbridge, Inc. (http://blackbridge.com), 
and Photo Science (Raber, 2012). 


DMC International Imaging Ltd. 
SLIM-6 and NigeriaSat-2 


The DMC Consortium satellites are owned and oper- 
ated by different nations in the consortium. They are 
placed in phased Sun-synchronous orbits to achieve 
daily revisit capability. The consortium, established in 
2003, has regular additions of new member satellites. 


SLIM-6 

The DMC SLIM.-6 linear array pushbroom sensor sys- 
tem has a 22 Xx 22 m spatial resolution with three multi- 
spectral bands (green, red, and near-infrared) that are 
Landsat ETM™ equivalent. It is a medium-resolution 
remote sensing system that is cross-calibrated against a 
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TABLE 2-14 Sensor characteristics of DigitalGlobe, Inc., QuickBird satellite; WorldView-1 and WorldView-2 satellites. 


DigitalGlobe, Inc. 
QuickBird (2001) WorldView-1 (2007) WorldView-2 (2009) 
Band Spectral Spatial Band Spectral Spatial Band Spectral Spatial 
Resolution Resolution Resolution Resolution Resolution Resolution 
(nm) (m) at Nadir (um) (m) at Nadir (ym) (m) at Na- 
dir 
1 430 —545 2.62 x 2.62 1 397 — 905 0.5 x 0.5 1 396 — 458 1.85 x 1.85 
2 466 — 620 2.62 x 2.62 2 442-515 1.85 x 1.85 
3 590-710 2.62 x 2.622 3 506 — 586 1.85 x 1.85 
4 715-918 2.62 x 2.62 4 584 — 632 1.85 x 1.85 
Pan 405 — 1053 0.65 x 0.65 5 624 — 694 1.85 x 1.85 
6 699 -749 1.85 x 1.85 
7 765-901 1.85 x 1.85 
8 856-1043 | 1.85 x 1.85 
Pan 447 — 808 0.46 x 0.46 
Sensor Linear array pushbroom Linear array pushbroom Linear array pushbroom 


reference standard satellite (Landsat 7) over the Libya 
4 site and Dome C Antarctica (DMC, 2014a). 


DMC-NigeriaSat-2 

DMC NigeriaSat-2 has a 2.5 x 2.5 m panchromatic 
band and four multispectral bands at 5 x 5 m spatial 
resolution. The swath width is 20.5 km (DMC, 2014b). 
It is pointable with a 2-day revisit to anywhere on the 
globe. 


Imaging Spectrometry Using 
Linear and Area Arrays 


In the past, most remotely sensed data were acquired 
in 4 to 12 spectral bands. Imaging spectrometry makes 
possible the acquisition of data in hundreds of spectral 
bands simultaneously. Because of the very precise na- 
ture of the data acquired by imaging spectrometry, 
more Earth resource problems can be addressed in 
greater detail. 


The value of an imaging spectrometer lies in its ability 
to provide a high-resolution reflectance spectrum for 
each picture element in the image. The reflectance 
spectrum in the region from 0.4 — 2.5 um can be used 
to identify a large range of surface cover materials that 


cannot be identified with broadband, low spectral reso- 
lution imaging systems such as the Landsat MSS, TM, 
and SPOT. Many surface materials have diagnostic ab- 
sorption features that are only 10 to 30 nm wide. 
Therefore, spectral imaging systems that acquire data 
in contiguous 10-nm bands may produce data with suf- 
ficient resolution for the direct identification of those 
materials with diagnostic spectral absorption features. 
For example, Figure 2-49 depicts high spectral resolu- 
tion crop spectra over the interval 400 to 1,000 nm ob- 
tained using an Imaging Spectrometer for an 
agricultural area near Bakersfield, CA. The absorption 
spectra for the Pima and Royale cotton differ from one 
another from about 725 nm, where the “red edge” is lo- 
cated, to about 900 nm, leading to the possibility that 
species within the same crop type might be distinguish- 
able. The Landsat and SPOT sensors, which have rela- 
tively large bandwidths, may not be able to resolve 
these spectral differences. 


Two approaches to imaging spectrometry are shown in 
Figures 2-9ef. The whiskbroom scanner linear array 
approach (Figure 2-9e) is analogous to the scanner ap- 
proach used for ETM”, except that radiant flux from 
within the IFOV is passed on to a spectrometer, where 
it is dispersed and focused onto a linear array of detec- 
tors. Thus, each pixel is simultaneously sensed in as 
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FIGURE 2-47 a) The ImageSat International EROS B satellite collects panchromatic (500 - 900 nm) imagery at 0.7 x 0.7 m 
spatial resolution. b) EROS B image of a bridge collapse in Minneapolis, MN (images courtesy of ImageSat International, NV). 
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FIGURE 2-48 a) The constellation of five RapidEye satellites. b) Spectral resolution of the RapidEye sensors including the 
special red-edge band. c) RapidEye image of San Diego, CA, obtained on June 18, 2011 (images courtesy of BlackBridge). 
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FIGURE 2-49 Imaging spectrometer crop spectra for Pima 
cotton, Royale cotton, and road surface extracted from 2 x 
2 m data obtained near Bakersfield, CA. 


many spectral bands as there are detector elements in 
the linear array. Because of high spacecraft velocities, 
orbital imaging spectrometry might require the use of 
two-dimensional area arrays (Figure 2-9f). This elimi- 
nates the need for the optical scanning mechanism. In 
this situation, there is a dedicated column of spectral 
detector elements for each linear array cross-track pix- 
el in the scene. 


Thus, traditional broadband remote sensing systems 
such as Landsat MSS and SPOT HRV undersample the 
information available from a reflectance spectrum by 
making only a few measurements in spectral bands up 
to several hundred nanometers wide. Conversely, imag- 
ing spectrometers sample at close intervals (bands on 
the order of tens of nanometers wide) and have a suffi- 
cient number of spectral bands to allow construction 
of spectra that closely resemble those measured by lab- 
oratory instruments. Analysis of imaging spectrometer 
data allows extraction of a detailed spectrum for each 
picture element in the image (Figure 2-49). Such spec- 
tra often allow direct identification of specific materials 
within the IFOV of the sensor based upon their reflec- 
tance characteristics, including minerals, atmospheric 
gases, vegetation, snow and ice, and dissolved matter in 
water bodies. 


Analysis of hyperspectral data often requires the use of 
sophisticated digital image processing software (dis- 
cussed in Chapter 11). This is because it is usually nec- 
essary to calibrate (convert) the raw hyperspectral 
radiance data to scaled reflectance before it can be 
properly interpreted. This means removing the effects 
of atmospheric attenuation, topographic effects (slope, 
aspect), and any sensor anomalies. Similarly, to get the 
most out of the hyperspectral data it is usually neces- 
sary to use algorithms that 1) allow one to analyze a 
typical spectra to determine its constituent materials, 


and 2) compare the spectra with a library of spectra 
obtained using handheld spectroradiometers such as 
those provided by the U.S. Geological Survey. 


Government agencies and commercial firms have de- 
signed hundreds of imaging spectrometers capable of 
acquiring hyperspectral data. It is beyond the scope of 
this book to list them all. A few satellite hyperspectral 
instruments are briefly discussed, including: NASA’s 
EO-1 Hyperion Hyperspectral Imager, NASA JPL’s 
Airborne Visible/Infrared Imaging Spectrometer, NA- 
SA’s Moderate Resolution Imaging Spectrometer, and 
NASA’s HyspIRI satellite. Two commercial hyperspec- 
tral sensors are discussed including: Itres, Inc. CASI- 
1500, and HyVista, Inc., HyMap, Inc. Additional in- 
formation about several of these sensors is provided in 
Chapter 11. 


NASA EO-1 Hyperion Hyperspectral 
Imager 


NASA’s Earth Observation (EO-1) satellite carried the 
Hyperion hyperspectral remote sensing system. 


Hyperion 

The Hyperion sensor provides high-quality calibrated 
data that can support evaluation of hyperspectral tech- 
nology for Earth observing missions. Hyperion is a 
pushbroom instrument (Figure 2-9f) with a swath 
width of 7.5 km. Hyperion has a single telescope and 
two spectrometers, one visible/near infrared (VNIR) 
spectrometer and one short-wave infrared (SWIR) 
spectrometer. A dichroic filter in the system reflects the 
energy from 400 to 1,000 nm to the VNIR spectrome- 
ter and transmits the energy from 900 — 2,500 nm to 
the SWIR spectrometer. There are 198 useful bands 
out of 220 that are calibrated, including channels 8—57 
for the VNIR and channels 77-224 for the SWIR 
(http://eol.usgs.gov/faq). The SWIR overlap with the 
VNIR from 900 — 1,000 nm allows cross calibration be- 
tween the two spectrometers. The spectral range of the 
instrument extends from 400 — 2,500 nm with a spec- 
tral resolution of 10 nm (Table 2-15). The Hg:Cd:Te 
detectors in the SWIR spectrometer are cooled by an 
advanced TRW cryocooler. The data typically are pro- 
cessed into cubes (19.8 km long by 7.5 km wide) to fa- 
cilitate data handling in current desktop computers. 
Each cube consists of 75 MB of data. A typical acqui- 
sition consists of multiple cubes. 


NASA Airborne Visible/Infrared 
Imaging Spectrometer (AVIRIS) 
The first airborne imaging spectrometer (AIS) was 


built to test the concept with infrared area arrays (Vane 
and Goetz, 1993). The spectral coverage of the instru- 
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TABLE 2-15 Characteristics of the NASA EO-1 Hyperion hyperspectral sensor, NASA’s Airborne Visible/Infrared Imaging 
Spectrometer (AVIRIS), ITRES Research, Ltd. CASI-1500, and HyVista’s, Inc. HyMap. 


Spectral Spectral Quantiza- IFOV Total 
Resolution Interval Number of tion (mrad) field of 
Sensor Technology (nm) (nm) Bands (bits) view (°) 
Hyperion Linear array 400 — 2,500 10 220 11 
AVIRIS Whiskbroom 400 — 2,500 10 224 12 1.0 30° 
linear array 
CASI-1500 Linear array 370 — 1,050 Peg 288 possible; the 14 0.49 40° 
(1,500) and number of bands 
area array CCD and the number of 
(1,500 x 288) pixels in the across- 
track are program- 
mable. 
HyMap Whiskbroom 450 — 2,480 13-17 128 25 Clas) 
linear array 


ment was 1.9 — 2.1 um in the tree mode and 1.2 — 2.4 
um in rock mode in contiguous bands that were 9.3 nm 
wide. 


AVIRIS 

To acquire data with greater spectral and spatial cover- 
age, AVIRIS was developed at NASA’s Jet Propulsion 
Laboratory in Pasadena, CA (Table 2-15). Using a 
whiskbroom scanning mirror and linear arrays of sili- 
con (Si) and indium-antimonide (InSb) configured as 
in Figure 2-9e, AVIRIS acquires images in 224 bands, 
each 10 nm wide in the 400 — 2,500 nm region (NASA 
AVIRIS, 2014). The sensor is typically flown onboard 
the NASA/ARC ER-2 aircraft at 20 km above ground 
level and has a 30° total field of view and an instanta- 
neous-field-of-view of 1.0 mrad, which yields 20 x 20 
m pixels. The data are recorded in 12-bits (values from 
0 to 4,095). 


Moderate Resolution Imaging 
Spectrometer (MODIS) 


The Moderate Resolution Imaging Spectrometer (MO- 
DIS) is flown on NASA’s EOS Terra (a.m. equatorial 
crossing time) and Agua (p.m. equatorial crossing 
time) satellites (Table 2-15). MODIS provides long- 
term observations to obtain knowledge about global 
dynamics and processes occurring on the surface of the 
Earth and in the lower atmosphere (NASA MODIS, 
2014). MODIS yields information on atmospheric 
(cloud cover and associated properties), oceanic (sea- 
surface temperature and chlorophyll), and land-surface 
(land-cover changes, land-surface temperature, and 
vegetation properties) characteristics. 


MODIS is in a 705-km Sun-synchronous orbit. It 
views the entire surface of the Earth every | to 2 days. 
It has a field of view of +55° off-nadir, which yields a 
swath width of 2,330 km (Figure 2-50a). MODIS ob- 
tains high radiometric resolution images (12-bit) of 
daylight-reflected solar radiation and day/night ther- 
mal emission over all regions of the globe. MODIS is a 
whiskbroom scanning imaging radiometer consisting 
of a cross-track scan mirror, collecting optics, and a set 
of linear detector arrays with spectral interference fil- 
ters located in four focal planes. MODIS collects data 
in 36 co-registered spectral bands: 20 bands from 0.4 — 
3 um and 16 bands from 3 — 15 um. The spatial and 
spectral resolution of the bands and their primary uses 
are summarized in Table 2-16. 


MODIS’ coarse spatial resolution ranges from 250 x 
250 m (bands | and 2) to 500 x 500 m (bands 3 
through 7) and 1 x 1 km (bands 8 through 36). A MO- 
DIS image of the Nile Delta, the Nile River, and much 
of the area surrounding the Red Sea is shown in Figure 
2-50b. MODIS provides daylight reflection and day/ 
night emission spectral imaging of any point on Earth 
at least every 2 days, with a continuous duty cycle. 


MODIS has one of the most comprehensive calibra- 
tion subsystems ever flown on a satellite remote sens- 
ing system. The calibration hardware includes a solar 
diffuser, a solar diffuser stability monitor, a spectrora- 
diometric calibration instrument, a black body for 
thermal calibration, and a space viewport. The calibra- 
tion allows the optical MODIS data to be converted 
into scaled percent reflectance. MODIS data are being 
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TABLE 2-16 Characteristics of the Terra satellite Moderate Resolution Imaging Spectrometer (MODIS). 


Band Spectral Resolution Spatial Resolution Band Utility 
(um) 

1 0.620 - 0.670 250 x 250 m Land-cover classification, chlorophyll absorp- 
2 0.841 - 0.876 250 x 250 m tion, leaf-area-index mapping 
3 0.459 - 0.479 500 x 500 m Land, cloud, and aerosol properties 
4 0.545 — 0.565 500 x 500 m 
5 1.230 - 1.250 500 x 500 m 
6 1.628 - 1.652 500 x 500 m 
i, 2.105 — 2.155 500 x 500 m 
8 0.405 - 0.420 1x 1km Ocean color, phytoplankton, biogeochemistry 
g 0.438 — 0.448 1x 1km 

10 0.483 - 0.493 1x 1km 

11 0.526 — 0.536 1x 1km 

12 0.546 — 0.556 1x 1km 

13 0.662 — 0.672 1x 1km 

14 0.673 — 0.683 1x 1km 

15 0.743 - 0.753 1x 1km 

16 0.862 — 0.877 1x 1km 

17 0.890 - 0.920 1x 1km Atmospheric water vapor 

18 0.931 - 0.941 1x 1km 

19 0.915 — 0.965 1x 1km 

20 3.600 — 3.840 1x 1km Surface—cloud temperature 

21 3.929 — 3.989 1x 1km 

22 3.929 — 3.989 1x 1km 

23 4.020 — 4.080 1x 1km 

24 4.433 - 4.498 1x 1km Atmospheric temperature 

25 4.482 — 4.549 1x1km 

26 1.360 — 1.390 1x1km Cirrus clouds 

27 6.535 - 6.895 1x 1km Water vapor 

28 7.175 — 7.475 1x 1km 

29 8.400 — 8.700 1x 1km 

30 9.580 — 9.880 1x1km Ozone 

31 10.780 — 11.280 x 1km Surface—cloud temperature 

32 11.770 - 12.270 1 km 

33 13.185 — 13.485 1x 1km Cloud-top altitude 

34 13.485 — 13.785 1x 1km 

35 13.785 — 14.085 1x 1km 

36 14.085 - 14.385 1x 1km 


processed to create numerous global datasets, includ- NASA Hyperspectral Infrared Imager 
ing (NASA MODIS, 2014): 


(HyspIRI) 


* land ecosystem variables (e.g., vegetation indices, | NASA’s Hyperspectral Infrared Imager (HyspIRI) is 
LAI, fraction of photosynthetically active radiation, to be launched in the future. It is designed to study the 


vegetation net primary production), 


¢ atmospheric variables (e.g., cloud fraction, cloud 
optical thickness, aerosol optical depth, etc.), and 


world’s ecosystems and provide information on natural 
disasters, such as volcanoes, wildfires and drought, and 
vegetation health. The sensor was recommended in the 
National Research Council Decadal Survey requested 


* ocean variables (e.g., sea-surface temperature and by NASA, NOAA, and USGS. The HyspIRI mission 


chlorophyll). 


includes two instruments mounted on a satellite in low 
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Moderate Resolution Imaging Spectrometer (MODIS) Imagery 


a. Three swaths of MODIS imagery over 
northern Africa on February 29, 2000. 


b. MODIS image of the Nile Delta 
obtained on February 28, 2000. 


FIGURE 2-50 a) A MODIS composite image for the North African Continent on February 29, 2000. A dust storm can be seen 
blowing off the coast of Morocco in the northwest corner. The swath width is 2,330 km. b) MODIS image of the Nile Delta on 


February 28, 2000 (images courtesy of NASA Visible Earth). 


Earth orbit (NASA HyspIRI, 2014). There is an imag- 
ing spectrometer measuring from the visible to short- 
wave infrared (VSWIR: 380 nm — 2,500 nm) in 10 nm 
contiguous bands and a multispectral imager measur- 
ing from 3 — 12 um in the mid and thermal infrared 
(TIR). The TIR sensor will have eight bands. Both 
VSWIR and TIR instruments have a spatial resolution 
of 60 m at nadir. The VSWIR will have a revisit of 19 
days and the TIR will have a revisit of 5 days. HyspIRI 
also includes an Intelligent Payload Module (IPM) 
which will enable direct broadcast of a subset of the 
data (Glavich et al., 2009). 


Itres, Inc. Compact Airborne 
Spectrographic Imager-1500 


ITRES Research, Ltd., of Canada has developed an 
innovative variety of innovative visible near-infrared 
(CASI-1500), short-wavelength infrared (SASI-600), 
mid-wavelength infrared (MASI-600), and thermal in- 
frared (TASI-600) hyperspectral instruments. 


CASI-1500 

CASI-1500 is a pushbroom imaging spectrometer 
based on the use of a 1,500 < 288 area array CCD. The 
instrument operates over a 670-nm spectral range (380 
— 1,050 nm) and has a 40° total field of view across 
1,500 pixels (Itres CASI-1500, 2014). A single line of 
terrain 1,500 pixels wide perpendicular to the flight 
path is sensed by the spectrometer optics (Figure 2-51). 
The incoming radiant flux from each pixel is then spec- 
trally dispersed along the axis of the area array CCD 


(Figure 2-51) so that a spectrum of energy (from blue 
through near-infrared) is obtained for each pixel across 
the swath. By repetitively reading the contents of the 
area array CCD as the aircraft moves along the flight 
path, a two-dimensional image at high spectral resolu- 
tion is acquired. Because the radiant flux for all pixels 
in a particular swath are recorded simultaneously, spa- 
tial and spectral co-registration is assured. The across- 
track spatial resolution is determined by the altitude of 
the CASI above ground level and the IFOV, while the 
along-track resolution depends upon the velocity of 
the aircraft and the rate at which the CCD is read. 


CASI-1500 is spectrally programmable meaning that 
the user can specify which bands are to be collected for 
a specific application (e.g., vegetation biomass moni- 
toring, bathymetric mapping, inventorying chlorophyll 
a concentration, urban/suburban analysis). The result 
is a programmable area array VNIR hyperspectral re- 
mote sensing system. 


SASI-600 

SASI-600 collects 100 bands of SWIR hyperspectral 
data from 950 — 2,450 nm at 15-nm intervals with 600 
across-track pixels. SASI-600 data are especially useful 


for geological exploration and plant speciation (Itres 
SASI-600, 2014). 


MASI-600 


MASI-600 is the first commercially available mid- 
wavelength hyperspectral sensor designed specifically 
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Hyperspectral Data Collection Using Linear and Area Arrays 
ITRES Research Ltd., Calibrated Airborne Spectrographic Imager (CASI) 1500 


NIR 
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inear array is dispersed onto the 
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the area array. 


b. CASI-1500 false-color composite of an area in Japan. 


FIGURE 2-51 a) Hyperspectral data collection using a linear array pushbroom sensor that distributes the energy onto a 
1,500 x 288 element area array sensitive to the wavelength region from 380 to 1,050 nm. b) CASI-1500 image of an area 


in Japan (image courtesy of ITRES Research, Ltd.). 


for airborne use. It records 64 bands in the 4 — 5 um 
range with a 40° across-track field of view encompass- 
ing 600 pixels (Itres MASI-600, 2014). 


TASI-600 

The TASI-600 is a pushbroom hyperspectral thermal 
sensor system designed specifically for airborne use. It 
collects 32 or 64 bands in the region from 8 — 11.5 um 


with a 40° across-track field of view encompassing 600 
pixels (Itres TASI-600, 2014). 


HyVista, Inc., HyMap 


HyVista Inc. of Australia operates a HyMap whisk- 
broom hyperspectral scanner (Figure 2-52) manufac- 
tured by Integrated Spectronics Pty Ltd. (HyVista 
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HyVista, Inc., HyMap™ Hyperspectral Remote Sensing 


Rotating mirror 


Spectral Range 
SWIR 2 | 1,950 — 2,480 nm 
SWIR 1 | 1,400 —- 1,800 nm 
NIR 890 — 1,350 nm 
Red 
Green 450 — 890 nm 
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128 bands 


2.0 mr across track 


FIGURE 2-52 HyMap hyperspectral remote sensing using whiskbroom technology in the region from 450 - 2,480 nm. 


HyMap, 2014). The HyMap sensor records 128 bands 
in the reflective region from 450 — 2,480 nm with con- 
tiguous spectral coverage (except in the atmospheric 
water vapor bands near 1,400 and 1,900 nm) with aver- 
age spectral sampling intervals between 13 — 17 nm. 
Typical ground spatial resolution is between 2 and 10 
m depending upon flight altitude AGL. 


Airborne Digital Cameras 


Digital cameras have revolutionized airborne photo- 
grammetric data collection. The imaging sensor of a 
digital camera is usually a charge-coupled device 
(CCD) or Complementary Metal-Oxide-Semiconduc- 
tor (CMOS) computer chip. The sensor detectors con- 
vert light into electrons that can be measured and 
converted into a radiometric intensity value. Digital 
cameras may be based on linear or area arrays of de- 
tectors (Figure 2-9b). 


Digital cameras utilize a lens with its associated dia- 
phragm to control the f/stop, a shutter to control the 
length of exposure, and a focusing device. However, the 
major difference is that instead of using film, CCD lin- 
ear or area arrays are located at the film plane. The lens 
focuses the light from the outside world onto the lin- 
ear- or area-array bank of detectors. The photons of 
light illuminating each of the detectors cause an electri- 
cal charge to be produced that is directly related to the 
amount of incident radiant energy. This analog signal 


is then sampled electronically and converted into a dig- 
ital brightness value ranging from 8-bit (values from 0 
— 255) to 12-bit (values from 0 to 4,095). The bright- 
ness values obtained from the analog-to-digital (A-to- 
D) conversion may be stored and read by computer 
systems. The CCDs are more sensitive to spectral re- 
flectance changes in the scene than the silver halide 
crystals used in conventional analog color and color- 
infrared aerial photography. 


Small-Format Digital Cameras 


These cameras typically collect digital images that are 
< 16 megapixels (MP) per band. For example, the Can- 
on digital camera shown in Figure 2-53b is based on an 
area array with 3,456 columns and 2,304 rows yielding 
approximately 8 MP per band. At the instant of expo- 
sure, the camera rapidly records three versions of the 
scene using internal filtration. The result is one image 
based solely on blue light reflected from the terrain, 
another based on only green light reflected from the 
terrain, and a final image produced only from reflected 
red light. The three individual black-and-white images 
are recorded in the camera’s random-access memory 
(RAM) and can be color-composited using additive 
color theory to produce a natural-looking color photo- 
graph. It is also possible to make the detectors sensitive 
to near-infrared light. 


The inexpensive unmanned aircraft system (UAS) 
shown in Figure 2-53ab contains an intervalometer to 
obtain photographs at specific intervals to obtain the 
necessary end-lap. The images are stored on a data re- 
corder. The operator on the ground knows where the 
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Inexpensive Unmanned Aerial System (UAS) using a Small-Format Digital Frame Camera 


a. Unmanned aerial system fuselage. 


c. Citrus grove near Weslaco, TX. 


Data recorder and 
intervalometer 


Digital camera 


d. Water treatment plant on South Padre Island, TX. 


FIGURE 2-53 a) An unmanned aerial system (UAS) modified to carry a small-format digital camera. b) System components 
include a data recorder, intervalometer, and GPS antenna (not shown). cd) Examples of aerial photography obtained using 
the unmanned aerial system (courtesy of Perry Hardin and Mark Jackson, Department of Geography, Brigham Young Univer- 


sity). 


plane is based on the use of an onboard GPS antenna 
(not shown). The inexpensive digital camera obtains 
high-resolution aerial imagery because it is usually 
flown relatively close to the ground (Figures 2-53cd). 


Medium-Format Digital Cameras 


These cameras are usually based on linear or area ar- 
rays with greater than 4,000 x 4,000 detectors per band 
(yielding 16 MP). 


Leica Geosystems, Ag., RCD30 

Leica Geosystems, Ag., produces several of the most 
popular medium- and large-format remote sensing dig- 
ital data-collection sensor systems used by many pho- 
togrammetric engineering firms in the United States 
(Leica, 2014a). For example, much of the imagery ac- 
quired for public use (e.g., USDA National Agriculture 
Imagery Program [NAIP]), counties, states) is collected 
using Leica digital cameras. The RCD30 provides mul- 
tispectral data in four co-registered bands (RGB and 
NIR) with each digital frame of imagery being 8,956 x 
6,708 pixels in dimension. The camera has forward 
motion compensation (FMC) along two axes (forward 


and lateral). It is one of the most accurate and most re- 
liable medium-format cameras available (Leica 
RCD30, 2014). 


Large-Format Digital Cameras 


Large-format digital cameras typically have very large 
CCD linear or area CCD arrays. These cameras sur- 
pass the performance of large-format analog frame 
cameras. Because of the large number of large-format 
analog cameras already in existence, digital cameras 
will be used along with the traditional analog cameras 
for some time. Eventually, however, most photogram- 
metric aerial photography will be obtained using large- 
format digital metric cameras. Two companies that 
produce some of the most important medium- to large- 
scale digital cameras are Leica Geosystems, Ag, and 
Microsoft Ultramap, Inc. 


Leica Geosystems, Ag., ADS80, Z/I Imaging 
DMC Acrial Photography 

The Leica Airborne Digital Sensor—ADS80 uses lin- 
ear array technology to collect data in panchromatic, 
red, green, blue (RGB) and near-infrared (NIR) bands. 
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FIGURE 2-54 a) Leica Geosystems Airborne Digital Sensor—ADS-80. b) Dispersion of incoming panchromatic, blue, green, 
red, and near-infrared energy onto the 12,000 detectors of each of the linear arrays. c) Orientation of the aft, Nadir, and for- 
ward-looking linear arrays (images courtesy of Leica Geosystems). 


Each of the CCD linear arrays consists of 12,000 pix- 
els. Each of the pixels in the linear arrays is 6.5 um in 
size (Leica, 2014a). The data are collected using the na- 
dir, aft- and forward-looking logic shown in Figure 2- 
54. All of the panchromatic and spectral data are in 
near-perfect registration at the same radiometric reso- 
lution (Leica ADS80, 2014) 


The Z/1 Imaging Digital Mapping Camera (DMC) 
family includes four different configurations that col- 
lect metric digital frame camera imagery. The base Z/1 
RMK D (5,760 x 6,400 pixels) is a four-camera-head 
multispectral sensor that collects RGB and NIR imag- 


ery. The Z/I DMC II140 consists of a five-camera-head 
multispectral sensor (RGB, NIR) (each 6,846 x 6,096 
pixels) plus a 140 megapixel panchromatic sensor 
(12,096 x 11,200 pixels) that doubles the ground reso- 
lution of the Z/I RMK D. The Z/1 DMC II230 is a 
five-camera-head multispectral sensor (RGB, NIR) 
(each 6,846 x 6,096 pixels) that has a 230 megapixel 
CCD (15,552 x 14,144 pixels) that increases ground 
coverage even more to reduce flight lines. The Z/1 
DMC II250 is a five-camera-head multispectral sensor 
(RGB, NIR) (each 6,846 x 6,096 pixels) that has a pan- 
chromatic band (16,768 x 14,016 pixels). It has longer 
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UltraCam™ Digital Frame Camera 
Used to Obtain Bing Maps Aerial Photography 


Panchromatic 


b. UltraCam natural color digital photography. — c. UltraCam color-infrared digital photography. 


FIGURE 2-55 Characteristics of the Microsoft, Inc. UltraCam™ digital frame camera. a) Lenses associated with panchromatic, 


color, and color-infrared data collection. b-c) Examples of natural color and color-infrared UltraCam™ 


aerial photography 


(images courtesy of Microsoft Photogrammetry Division, www.iflyultracam.com; updated from Jensen et al., 2012). 


focal length optics, yielding even greater coverage with 
finer detail at lower flight altitudes (Leica Z/1, 2014). 


Microsoft, Inc., UltraCam Eagle 

The Microsoft, Inc. Photogrammetry Division devel- 
oped the UltraCam digital frame camera primarily to 
obtain imagery for its Bing Maps platform search en- 
gine. In addition, the division markets UltraCam cam- 
eras and related products to the photogrammetric 
engineering industry, including: UltraCam Eagle (the 
newest sensor), ULTRACAM Xp, ULTRACAM Xp- 
WA, and ULTRACAM Lp (Microsoft UltraCam, 
2014). 


UltraCam imagery is the data source for automated 
3D framework production in Bing Maps (Wiechert, 
2009a). They use 80% endlap and 60% sidelap Ultra- 


Cam photography (yielding up to 12 rays per ground 
pixel based on multi ray photogrammetry) to extract 
extremely high-resolution digital surface models 
(DSM) of the terrain with >50 points per m2 (Wiechert 
2009b). Derived DSM and digital terrain models 
(DTM) in conjunction with image classification (con- 
sisting of 10-15 classes such as buildings, vegetation, 
concrete, etc.) are used to create the three-dimensional 
roof polygons, extrude completed roof geometry to the 
DTM, and apply photo texture from UltraCam images 
to create a three-dimensional Virtual Earth served in 
Bing Maps (Wiechert, 2009ab). UltraCam photogram- 
metry is performed using the ULTRAMAP Workflow 
Software System. 


The UltraCam Eagle system has one of the largest 
panchromatic image footprints when compared with 


Pictometry 
Digital Aerial Photography 
of Columbia, SC 
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Photography was obtained 
from multiple flightlines. 


FIGURE 2-56 Natural color vertical and oblique aerial photography of the Strom Thurmond Wellness Center in Columbia, 
SC, obtained at four cardinal directions (North, East, South, West) and at Nadir (i-e., vertical). The nominal spatial resolution 
was 6 x 6 in. (photography courtesy of Pictometry International, Inc.). 


the other ultra-large format metric camera with 20,010 
x 13,080 detectors. This can reduce the number of 
data-collection flight lines required for a given project. 
Multispectral data are collected in the blue, green, red, 
and reflective near-infrared regions of the spectrum us- 
ing sensors with 6,67 x 4,360 detectors. Radiometric 
resolution is 8- to 16-bits (Microsoft UltraCam, 2014). 
An example of the quality of UltraCam Eagle multi- 
spectral digital frame camera photography is shown in 
Figure 2-55. The frame data-collection rate is 1.8 sec- 
onds which allows 80% to 90% forward overlap if de- 
sired. Microsoft markets other digital frame cameras 
(not discussed here) (Microsoft UltraCam, 2014). 


Digital Oblique Aerial Photography 


Sometimes people feel more comfortable looking at 
and analyzing oblique aerial photography rather than 
vertical aerial photography (Figure 2-56). Several com- 


mercial vendors now collect vertical as well as oblique 
aerial photography. 


Pictometry International, Inc. Oblique and 
Vertical Aerial Photography 

Pictometry uses a five-camera arrangement whereby 
every time a vertical aerial photograph is collected, 
four additional oblique photographs are obtained in 
the North, East, South, and West directions (Pictome- 
try, 2014). The flight lines overlap by 20 to 30%, which 
allows each feature in the landscape such as a building 
to be recorded and viewed from many different vantage 
points. The image analyst can simply select the view 
that is most pleasing and provides the most useful the- 
matic information. Pictometry aerial photography of a 
portion of Columbia, SC, is shown in Figure 2-56. In- 
novations in digital image processing make it possible 
to register the oblique aerial photography in a GIS 
where both horizontal and vertical measurements of 
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Digital Frame Camera Image of the Toquepala Copper Mine in 
Southern Peru Taken from the International Space Station 


FIGURE 2-57 Astronaut photograph of the Toquepala Copper Mine in Southern Peru taken from the International Space Sta- 
tion on September 22, 2003, with a Kodak DCS 760 digital camera (photo #ISSO07-E-15222). The open pit mine is 6.5 km 
across and descends more than 3,000 m into the earth. A dark line on the wall of the pit is the main access road to the bot- 
tom. Spoil dumps of material mined from the pit are arranged in tiers along the northwest lip of the pit (courtesy of NASA 
Earth Observatory and Dr. Kamlesh Lulla, NASA Johnson Space Center). 


structures and the terrain can be made while viewing 
the oblique photography. 


Satellite Digital Frame 
Camera Systems 


Despite the ongoing development of electro-optical re- 
mote sensing instruments, traditional optical camera 
systems continued to be used for space-survey purpos- 
es. For example, the U.S. Space Shuttle and Space Sta- 
tion astronauts routinely collected photography using 
Hasselblad and Linhof cameras. 


U.S. Space Shuttle Photography 


NASA astronauts routinely documented Earth pro- 
cesses during Space Transportation System (STS) mis- 
sions using both analog and digital camera systems. 
These efforts resulted in an impressive database of 
more than 400,000 Earth images. Photographic docu- 


mentation of Earth processes during manned space- 
flights was the cornerstone of the Space Shuttle 
Observations program, as it was with the earlier Mer- 
cury, Gemini, Apollo, and Skylab Earth observations 
programs (Lulla and Dessinov, 2000). During the 
space shuttle era, more than 250 selected sites of inter- 
est to geoscientists were identified. Data from these 
sites were acquired during Space Shuttle missions and 
cataloged into a publicly accessible electronic database 
according to the specific mission (e.g., STS-74) or by 
thematic topic (NASA Shuttle Photography, 2014). 


Space Shuttle Analog Cameras 

The primary analog cameras used during Space Shut- 
tle missions were the Hasselblad and Linhof systems. 
NASA-modified Hasselblad 500 EL/M 70-mm camer- 
as were used with large film magazines, holding 100 to 
130 exposures. 


The four windows in the aft part of the Space Shuttle 
were used to obtain photography of the Earth. The 
windows allowed only 0.4 to 0.8 um light to pass 
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Remote Sensing Data Formats 
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FIGURE 2-58 Selected digital image data formats. 


through. This resulted in the use of two primary film 
bases in the Hasselblad and Aero-Technika Linhof 
camera systems, including visible color (Kodak 5017/ 
6017 Professional Ektachrome) and color-infrared 
(Kodak Aerochrome 2443) films. 


Space Shuttle photographs were obtained at a variety 
of Sun angles, ranging from 1° to 80°, with the majori- 
ty of pictures having Sun angles of approximately 30°. 
Very low Sun angle photography often provides unique 
topographic views of remote mountainous areas other- 
wise poorly mapped. Sequential photographs with dif- 
ferent look angles can provide stereoscopic coverage. 
Seventy-five percent of the photographs in the archive 
cover the regions between 28°N and 28°S latitude, pro- 
viding coverage for many little-known tropical areas. 
Twenty-five percent of the images cover regions be- 
tween 30° to 60° N and S latitude. 


The Space Shuttle Earth Observations Project 
(SSEOP) photography database of the Earth Science 
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Branch at the NASA Johnson Space Center contains 
the records of more than 400,000 photographs of the 
Earth made from space. A select set of photographs 
has been digitized and placed in the public domain 
(http://images.jsc.nasa.gov). 


eee Shuttle and Space Station Digital 
Photography 

The International Space Station (ISS) was launched 
November 2, 2000. It continues the NASA tradition of 
Earth observation from human-tended spacecraft. The 
ISS U.S. Laboratory Module has a specially designed 
optical window with a clear aperture 50.8 cm in diame- 
ter that is perpendicular to the Earth’s surface most of 
the time. In 2001, Space Shuttle astronauts began ac- 
quiring digital images that were transmitted directly to 
the ground while in orbit. An astronaut photograph of 
the Toquepala Copper Mine in Southern Peru taken 
from the International Space Station on September 22, 
2003, with a Kodak DCS 760 digital camera is shown 
in Figure 2-57. 
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Digital Image Data Formats 


The image analyst can order digital remote sensor data 
in a variety of formats. The most common formats are 
(Figure 2-58): 


¢ Band Interleaved by Pixel (BIP), 
¢ Band Interleaved by Line (BIL), and 
¢ Band Sequential (BSQ). 


To appreciate the data formats, consider a hypothetical 
remote sensing dataset containing just nine pixels ob- 
tained over the interface between land and water (Fig- 
ure 2-58). The image consists of three bands (band | = 
green; band 2 = red; band 3 = near-infrared). The 
brightness value (B ee ¥ 4) row, column, and band nota- 
tion is provided (Figure 2-58a). 


Band Interleaved by Pixel Format 


The BIP format places the brightness values in bands 
associated with each pixel in the dataset in sequential 
order For example, if a dataset contained three bands 
the format for the first pixel in the matrix (1,1) is 1,1,1; 
1,1,2; 1,1,3. The brightness values for pixel (1,2) are 
then placed in the dataset (e.g., 1,2,1; 1,2,2; and 1,2,3), 
and so on. An end-of-file (EOF) marker is placed at 
the end of the dataset (Figure 2-58b). 


Band Interleaved by Line Format 


The BIL format creates a file that places the brightness 
values in 1 bands associated with each line in the data- 
set in sequential order. For example, if there are three 
bands in the dataset, all of the pixels in line 1, band 1 
are followed by all of the pixels in line 1, band 2, and 
then line 1, band 3. An EOF marker is placed at the 
end of the dataset (Figure 2-58c). 


Band Sequential Format 


The BSQ format places all of the individual pixel val- 
ues in each band in a separate and unique file. Each 
band has its own beginning header record and EOF 
marker (Figure 2-58d). 


Summary 


The public and scientists should perform digital image 
processing of remote sensor data only after they under- 
stand and appreciate exactly how the imagery was col- 
lected. This chapter provided information about the 


characteristics of many important digital remote sens- 
ing systems and the most often used digital data for- 
mats. 
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3 DIGITAL IMAGE PROCESSING 


Analog (hard-copy) and digital remotely sensed data 
are used operationally in many Earth science, social 
science, and planning applications (e.g., Jensen and 
Jensen, 2013). Analog remotely sensed data such as 
positive 9 x 9 in. aerial photographs are routinely ana- 
lyzed using the fundamental analog elements of image 
interpretation (e.g., size, shape, shadow) and optical in- 
struments such as stereoscopes and zoom-transfer- 
scopes. Digital remote sensor data are analyzed using a 
digital image processing system that consists of com- 
puter hardware and special-purpose image processing 
software. 


o. Overview 


This chapter begins with a review of computer hard- 
ware characteristics that are of value when conducting 
digital image processing, including: type of computer, 
central processing unit (CPU), system random-access 
(RAM) and read-only (ROM) memory, mass storage 
and data archive considerations, video display spatial 
and spectral resolution, input and output devices, etc. 
The hardware associated with typical digital image 
processing laboratories is discussed. 


High-quality digital image processing software is criti- 
cal for successful digital image processing. The soft- 
ware should be easy to use and functional. The most 
important digital image processing functions are intro- 
duced. Many of the most commonly used digital image 
processing systems are then reviewed including their 
functional strengths. Image processing system cost 
constraints are introduced. 


HARDWARE AND SOFTWARE 


High-quality digital image processing software has sig- 
nificant capabilities. However, there is often a need to 
perform some type of geospatial analysis that is not 
available in the standard image processing software. 
When this situation occurs, a well-trained digital image 
processing professional may be able to program new 
geospatial code that can function within the standard 
image processing software. Digital image processing 
software that allows users to create custom digital im- 
age processing applications is identified. 


Digital Image Processing 
Hardware Considerations 


To successfully process digital remote sensor data it is 
usually best to hire people who are trained in a system- 
atic body of knowledge (e.g., forestry, agronomy, urban 
planning, geography, geology, marine science) and who 
have considerable knowledge in GIScience (cartogra- 
phy/geodesy, remote sensing, and geographic informa- 
tion systems). They must understand the theoretical 
basis of remote sensing data-collection systems and the 
various digital image processing (and GIS) algorithms 
and how to properly apply the technology to their spe- 
cific systematic body of knowledge. It is the quality 
and creativity of the people, not the hardware or the 
software, that dictates how useful the digital image 
processing will be. 


Qualified image analysts select an appropriate digital 
image processing system that a) has a reasonable learn- 
ing curve and is relatively easy to use, b) has a reputa- 
tion for producing accurate results (ideally the 
software vendor has ISO certification), c) will produce 
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the desired results in an appropriate format (e.g., map 
products in a standard cartographic data structure 
compatible with most GIS), and d) is within their de- 
partment’s budget. Table 3-1 summarizes some of the 
important factors to consider when selecting a digital 
image processing system. It is useful to review these 
factors. 


Central Processing Unit 
Considerations 


Many Earth resource analysis and planning projects 
require that large geographic areas be inventoried and 
monitored through time. Therefore, it is common to 
obtain remotely sensed data of the study area. Unfor- 
tunately, the amount of digital data generated by a re- 
mote sensing system can be daunting. For example, a 
Landsat 5 Thematic Mapper 185 x 170 km scene con- 
sists of seven bands of 30 x 30 m data (the thermal 
channel was actually 120 x 120 but was resampled to 
30 x 30 m) and is approximately 244 Mb (5,666 rows x 
6,166 columns xX 7 bands / 1,000,000 = 244 Mb). 
Change detection studies require multiple dates of re- 
mote sensor data (Canty, 2014). 


Furthermore, many public agencies and Earth resource 
scientists are now taking advantage of even higher spa- 
tial and spectral resolution remote sensor data. For ex- 
ample, a single 11 x 11 km GeoEye, Inc. IKONOS-2 
scene of panchromatic data consisting of 1 x 1 m pixels 
is 121 Mb (11,000 x 11,000 / 1,000,000 assuming 8-bit 
pixels). A single 512 x 512 pixel AVIRIS hyperspectral 
subscene contains 224 bands of 12-bit data and is 88.1 
Mb (512 x 512 x 1.5 xX 2247/ 1,000,000). Processing 
such large remote sensor datasets requires a significant 
number of computations. The type of computer select- 
ed dictates how fast (efficient) the computations or op- 
erations can be performed and the precision with 
which they are made. 


History of Central Processing Units and 
Efficiency Measurement 

The central processing unit (CPU) is the computing 
part of the computer. It consists of a control unit and 
an arithmetic logic unit. The CPU: 


* performs numerical integer and/or floating point 
calculations, and 

¢ directs input and output from and to mass storage 
devices, color monitors, digitizers, plotters, etc. 


The CPU’s efficiency can be measured in terms of 
¢ the number of cycles it can process in one second, 


e.g., 3.7 GHz means the CPU performs approxi- 
mately 3.7 billion cycles per second, 


TABLE 3-1 Factors to consider when selecting a digital im- 
age processing system. 


Digital Image Processing System Considerations 


e Number and speed of the computer’s CPU(s) 

e¢ Operating system (e.g., Microsoft Windows, UNIX, 
Linux, Apple) 

e Amount of random-access memory (RAM), Video 
RAM, and graphics processors 

e Number of image analysts that can use the system at 
one time and the mode of operation (e.g., interactive 
or batch) 

¢ Serial and/or parallel processing capability 

e Arithmetic coprocessor or array processor 

e Software compiler(s) 

e Type of mass storage (e.g., hard disk, CD, DVD, flash 
drive) and amount (e.g., gigabytes) 

e Monitor display spatial resolution (e.g., 1024 x 768 
pixels) 

e Monitor color resolution (e.g., 24 bits of image pro- 
cessing video memory yields 16.7 million displayable 
colors) 

e Input devices (e.g., optical-mechanical drum or flatbed 
scanners, area array digitizers) 

¢ Output devices (e.g., CD, DVD, film-writers, ink-jet 
printers, dye sublimation printers) 

e¢ Networks (e.g., local area, wide area, Internet) 

e Image processing software (e.g., ERDAS Imagine, 
Environment for Visualizing Images, PCl Geomatica, 
ER MAPPER, IDRISI, Esri Image Analyst, eCognition, 
TNTmips, VIPER) 


e Interoperability with major GIS software 


¢ how many millions of instructions it can process per 
second (MIPS), e.g., 500 MIPS, and/or 


¢ the number of transistors used by the CPU. 


MIPS are difficult to compare between CPU architec- 
tures. In fact, MacNeil (2004) suggests that the term 
MIPS should probably be titled “misleading indicator 
of processor speed.” Currently, the most reliable mea- 
sure of the speed of a CPU is the number of transistors 
associated with the CPU. 


To appreciate the quality of the computers that we rou- 
tinely use for digital image processing today, it is in- 
structive to briefly review the history of the central 
processing unit and consider some of its characteris- 
tics. The ENIAC was the first computer. It was invent- 
ed in 1946 and weighed approximately 30 tons. In 
1968, there were only 30,000 computers in the entire 
world—mostly mainframes that occupied entire rooms 
and refrigerator-sized mini-computers. People pro- 
grammed the computers using punch cards. This 
changed when several people who worked for Fairchild 
Semiconductor left to start their own business—Intel, 
Inc. 
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TABLE 3-2 Historical development of Intel, Inc., central processing units for personal computers (Intel, 2014). 


Clock Speed Manufacturing 
Central (KHz) Technology 
Processing Unit (MHz) (um) 
(Date introduced) (GHz) Transistors (nm) Significance 
4004 (1971) 108 KHz 2,300 10 um First microcomputer chip 
8008 (1972) 500-800 KHz 3,500 10 um 2X as powerful as the 4004 
8080 (1974) 2 MHz 4,500 6 um Made video games and home com- 
puters possible 
8086 (1978) 5 MHz 29,000 3 um First 16-bit processor 
8088 (1979) 5 MHz 29,000 3 um Used in first IBM PC 
286 (1982) 6 MHz 134,000 1.5 um 3-6x performance of 8086 
386 (1985) 16 MHz 275,000 1.5 um First to process 32-bit data and run 
multiple programs at one time 
486 (1989) 25 MHz 1.2 million 1 um First integrated floating point unit 
Pentium (1993) 66 MHz 3.1 million 0.8 um Could now do speech, sound 
Pentium II (1997) 300 MHz 7.5 million 0.25 um MMX media enhancement 
Pentium III (1999) 500 MHz 9.5 million 0.18 um Low-power states conserve energy 
Pentium IV (2000) iL) Gaz 42 million 0.18 um Ushered in nanotechnology age 
Pentium M (2002) 1. Gre 55 million 90 nm Revolutionized portable computing 
Pentium D (2005) 3.2 GHz 291 million 65 nm First desktop duel-core processors 
Quad-Core Xeon (2007) 3.0 GHz 820 million 45 nm Micro architecture enhancements 
3rd and 4rd Generation 3-4 GHz >1.4 billion 22 nm Hyper-threading technology 
Core (2013-2014) 


In 1969, Nippon Calculating Machine Corporation 
asked Intel, Inc., to design computer chips for its new 
Busicom printing calculator. Intel designed a set of 
four chips known as the MCS-4. It included a central 
processing unit (CPU) chip—the 4004—as well as a 
supporting read-only memory (ROM) chip for the cus- 
tom applications programs, a random-access memory 
(RAM) chip for processing data, and a shift-register 
chip for the input/output (I/O) port. Intel subsequently 
purchased the rights from Nippon Calculating Ma- 
chine Corporation and launched the Intel® 4004 pro- 
cessor and its chipset with an advertisement in the 
November 15, 1971, issue of Electronic News (Table 3- 
2). The Intel® 4004 became the first general-purpose 


programmable CPU in the marketplace—a “building 
block” that engineers could purchase and then custom- 
ize with software to perform different functions in a 
wide variety of electronic devices (Intel, 2014). 


Gordon Moore was one of the founders of Intel, Inc. 
As early as April 19, 1965, he published an article in 
Electronics Magazine in which he made a prediction 
about the semiconductor industry. He realized that 
each new chip contained roughly twice as much capac- 
ity as its predecessor and each chip was released within 
18 to 24 months of the previous chip. If this trend con- 
tinued, he reasoned, computing power would rise ex- 
ponentially over relatively brief periods of time. 
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History of Intel Microprocessor Transistors 
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FIGURE 3-1 Moore’s law suggests a relationship between the capacity of computer chips (e.g., number of transistors) and the 
18- to 24-month release interval. a,b) Number of transistors in Intel microprocessors displayed in real numbers and exponen- 
tially. c) The 108-KHz 4004 Intel chip produced in 1971 had 2,300 transistors. d) The 1.5-GHz Pentium IV produced in 2000 had 
42 million transistors. e) The 3.7-GHz 3rd Generation processors have more than 1.4 billion processors (information courtesy 

of Intel, 2014). It is important to note that Intel, Inc., is not the only manufacturer of computer chips. 


Moore’s law described a trend that has continued and 
is still remarkably accurate. For example, the number 
of transistors in Intel chips from 1971 through 2012 
listed in Table 3-2 ranged from 2,300 to more than 1.4 
billion (Figure 3-1a). The same information is plotted 
logarithmically in Figure 3-1b revealing a straightfor- 
ward linear pattern. Photographs of the 4004, Pentium 
IV, and 3rd Generation CPUs are shown in Figures 3- 
Ic-e. 


Many personal computers, workstations, and especial- 
ly mainframe computers have multiple CPUs. A single 
CPU functions in a serial manner. The presence of 
multiple CPUs allows parallel processing to take place. 
The operating system and/or GIS software is able to 
parse (distribute) different tasks to the several CPUs, 
dramatically improving processing speed. 
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Personal Computers are Ideal for 
Digital Image Processing and GIS Analysis 


FIGURE 3-2 Properly configured personal computers and 
computer workstations are ideal for digital image processing 
and GIS analysis. In this example, the outline (and area) of 
several large buildings (shown in red) have been extracted 
from the natural color orthophoto (data courtesy State of 
Utah). 


Type of Computer 


All digital image processing system software vendors 
list the minimum system requirements necessary to run 
their software. Keep in mind that these are the mini- 
mum requirements. Computers with components that 
exceed the minimum requirements generally will be 
more useful and efficient when conducting digital im- 
age processing. If the computer has only the minimum 
hardware required to run the digital image processing 
software, the software will probably run slowly and will 
most likely dominate the operating system’s resources. 
This is especially a problem if you are a “multi-tasker” 
who frequently has other programs running at the 
same time as the digital image processing software 
(e.g., word processor, graphics program, Internet 
browser). 


Computers used for digital image processing can be or- 
ganized into three main categories: personal comput- 
ers (PCs), computer workstations, and mainframe 
computers. 


Personal Computers 

Personal computers (PCs) are the workhorses of the 
digital image processing industry (Figure 3-2). These 
include relatively inexpensive computers such as desk- 
tops, laptops, and tablets. Most of the personal com- 
puters now come with multiple CPUs. A multi-core 


processor is a single computing component with two or 
more independent central processing units (called 
cores), that read and execute program instructions. 
CPUs were originally developed with only one core. A 
dual-core processor has two cores (e.g., Intel Core 
Duo), a quad-core processor contains four cores (e.g., 
Intel quad-core 77), a hexa-core processor contains six 
cores (e.g., Intel Core 17 Extreme Edition), an octa- 
core processor contains eight cores and so on. 


Current PCs have much greater clock speeds, billions 
of transistors and can process instructions faster than 
their predecessors because they have 64-bit registers 
(word size) compared to the 8-bit registers used histor- 
ically. Many digital image processing—related compa- 
nies provide their employees with high-quality 
personal computers because of their low initial cost 
and inexpensive maintenance requirements. Interest- 
ingly, it seems that one can always purchase a “good” 
personal computer for digital image processing analy- 
sis for under $2,500. Ideally, the computer should have 
>8 GB of RAM, a large hard disk (>1 TB), a rewrit- 
able disk drive (e.g., DVD-RW), a precise graphic input 
device (e.g., a cursor), and a good graphics display sys- 
tem (monitor and video card). Common operating sys- 
tems suitable for personal computers include 
Microsoft Windows products (e.g., Windows 8), 
UNIX, Linux, and the Apple Macintosh OS. These 
operating systems allow computers to be networked 
and have access to the Internet. 


Computer Workstations 

Computer workstations usually contain more powerful 
processors, more RAM, larger hard disk drives, and 
very high-quality graphics display capability. These im- 
proved components allow workstations to perform dig- 
ital image processing analysis more rapidly than a 
personal computer. However, the cost of a workstation 
is usually two to three times more than the cost of a 
personal computer. The most common workstation 
operating systems are UNIX, Linux, and various M1- 
crosoft Windows products. 


Mainframe Computers 

Mainframe computers perform calculations more rap- 
idly than personal computers and workstations, and 
they are able to support hundreds of users simultane- 
ously. They may contain hundreds of CPUs in which 
case they are usually referred to as super computers. 
Mainframes are ideal for intensive CPU-dependent 
tasks such as overlay analysis, large database opera- 
tions, and raster rendering. If desired, the output from 
mainframe processing can be passed to a personal 
computer or workstation for less intensive processing. 
Mainframe computers are usually expensive to pur- 
chase and maintain. Also, digital image processing 
software for mainframe computers is more expensive. 
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Read-Only Memory and Random- 
Access Memory 


Computers used for digital image processing have sev- 
eral other characteristics that must be considered, such 
as Read-Only Memory (ROM) and Random-Access 
Memory (RAM). ROM retains information even after 
the computer is turned off because power is supplied 
by a battery that occasionally must be replaced. When 
a computer is turned on, the computer examines the 
information stored in the various ROM registers and 
uses this information to proceed. Most personal com- 
puters have sufficient ROM to perform quality digital 
image processing. 


RAM is the computer’s primary temporary workspace. 
Unlike ROM, the data stored in RAM are lost when 
the computer is turned off. Computers should have 
sufficient RAM for the operating system, image pro- 
cessing software, and any spatial data that must be held 
in temporary memory while calculations are per- 
formed. Because of this, the amount of RAM is one of 
the most important considerations when purchasing a 
computer for digital image processing. It seems that 
one cannot have too much RAM for digital image pro- 
cessing. It is good practice to have > 8 GB of RAM for 
digital image processing applications. Fortunately, 
RAM prices continue to decrease as the amount of 
RAM and its speed increase. 


Serial and Parallel Image Processing 


Most PCs, workstations, and mainframe computers 
now have multiple CPUs that operate concurrently 
(Figure 3-3). Some expensive parallel super computers 
have hundreds and even thousands of cores (e.g., IBM 
Blue Gene/Q). Specially written parallel processing 
software can parse (distribute) the remote sensor data 
to specific CPUs (cores) to perform digital image pro- 
cessing (e.g., Sorokine, 2007). This can be much more 
efficient than processing the data serially. For example, 
consider performing a per-pixel classification on a typ- 
ical 1,024 row by 1,024 column remote sensing dataset 
(Figure 3-3a). In the first example, each pixel is classi- 
fied by passing the spectral data associated with each 
pixel to the CPU, classifying it, and then progressing to 
the next pixel. This is inefficient serial processing (Fig- 
ure 3-3a). 


Conversely, suppose that instead of just one CPU we 
had 1,024 CPUs. In this case, the class of each of the 
1,024 pixels in row one could be determined using 
1,024 separate CPUs (Figure 3-3b). If this were the 
only task required in the procedure, the parallel image 
processing would classify the line of data approximate- 
ly 1,024 times faster than serial processing. In an en- 


tirely different parallel configuration, each of the 1,024 
CPUs could be allocated an entire row of the dataset. 


While parallel processing might appear to be the pana- 
cea to digital image processing efficiency, it is impor- 
tant to note that the potential speedup of an algorithm 
using parallel computing is constrained by Amdahl’s 
law (Amdahl, 1967). It states that if a is the fraction of 
running time a program spends on non-parallel parts 
(called sequential parts), then the maximum speedup 
of the process (S) using parallel code is: 
1 


SSS, 


. (3.1) 


For example, if the non-parallel (sequential) portion of 
a program accounts for 10% of the running time, we 
can get no more than a 10x speedup, regardless of how 
many processors are added. This is demonstrated in 
Figure 3-3c: 


If the non-parallel (sequential) portion of a program 
accounts for 50% of the running time, we can get no 
more than a 2X speedup, regardless of how many pro- 
cessors are used as shown in Figure 3-3c: 


=e Saee 
e950. °° 


Interestingly, all of the CPUs (cores) do not have to re- 
side on the same computer or even in the same city to 
perform parallel processing. It is possible to perform 
parallel processing by connecting individual computer 
systems via a network. This type of parallel processing 
requires sophisticated distributed processing software. 
In practice, it is difficult to parse a program so that 
multiple CPUs can efficiently execute different por- 
tions of the program without interfering with one an- 
other. Vendors continue to develop digital image 
processing code that takes advantage of multiple-core 
parallel architecture. 


Mode of Operation and User 
Interface 


Image analysts can process remotely sensed data inter- 
actively or in batch mode. Ideally the processing takes 
place in an interactive environment using a well-crafted 
graphical user interface (GUI). 


Mode of Operation 


Breakthroughs in analyst image understanding and 
scientific visualization are generally accomplished by 
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Serial versus Parallel Digital Image Processing 
to Perform Per-pixel Classification 
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FIGURE 3-3 Digital image classification using a) serial, and b) parallel image processing logic. There are 1,024 pixels in the 
line of data. In a serial processing environment, each pixel of each line in each band is processed sequentially using a single 
CPU. In a parallel processing environment, each of the 1,024 CPUs could be requested to process 1) each individual pixel in a 
line of remote sensor data containing 1,024 pixels (as shown), or 2) an entire line of data (not shown). c) A graphic representa- 
tion of Amdahl’s Law documenting the amount of speedup expected when a computer program consists of certain propor- 
tion of parallel code. For example, a program that consists of 10% sequential code and 90% parallel code can have a 
maximum speedup of 10x. 
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Computer Systems and Peripheral Devices in a Typical 
Digital Image Processing Laboratory 


Personal Computers: >4 GHz CPU, | GB Graphics RAM, 
>8 GB RAM, >1 TB hard disk, CD/DVD/Blu-ray Disk, Mouse 
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linear array digitizer 
(>2,000 dpi) 


Color laser printer 
(>2,000 dpi) 

Requirements 

- Graphical user interface (GUI) 

- Windows operating system 

- Compilers 

- Image processing software (e.g. ERDAS, 
ENVI, IDRISI, PCI, eCognition) 

- GIS software (e.g., ArcGIS) 

- Maintenance agreements on hardware, 

Laptop PC for in situ data software and the network 

logging and presentations 
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Ink-jet printer 
36 in. 


(1) Server, RAID mass storage 
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FIGURE 3-4 Digital image processing labs usually contain a number of relatively sophisticated PCs. In this example, there are 
six 24-bit color desktop personal computers (PCs), one laptop, and a server. The PCs communicate locally via a local area net- 
work (LAN) and with the world via the Internet. Each 4 GHz PC has >8 GB random-access memory, >1 TB hard disk space, 
and a CD, DVD and/or blue-ray disk. Image processing software and remote sensor data can reside on each PC (increasing 
the speed of execution) or be served by the server, minimizing the amount of mass storage required on each PC. Scanners, 
printers, and plotters are required for input of digital remote sensor data and output of important results. 
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Typical Digital Image Processing and GIS Computer Laboratory 


Color laser 
printer and 
scanner 


Overhead projector 
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FIGURE 3-5 This typical digital image processing and GIS computer laboratory consists of multiple high-end personal com- 
puters with single, very fast CPUs, >1 TB mass storage, high resolution computer screens (1,900 x 1,200) with 512 MB to 1 
GB of video memory, cursors, connection to a local area network (LAN) and access to the Internet. All of the computers have 
access to the color laser printer and scanner and to several devices not shown including a digitizing table and E-size ink-jet 
plotter. The digital overhead projector allows the instructor to display items of interest and perform real-time GIS analysis. 


placing the analyst as intimately in the image process- 
ing loop as possible and allowing his or her intuitive 
capabilities to take over. Ideally, every analyst has ac- 
cess to his or her own digital image processing system. 
Unfortunately, this is not always possible due to cost 
constraints. The sophisticated PC or workstation labo- 
ratory environment shown in Figure 3-4 would be ideal 
for six or seven people doing digital image processing 
in an educational or research environment. It would 
probably be ineffective for education or short course 
instruction where many analysts (e.g., >20) must be 
served. 


A real digital image processing/GIS computer labora- 
tory is shown in Figure 3-5. In this laboratory, the 
computers are connected via a local area network 
(LAN) that allows multiple users to access spatial data 
from a common source (e.g., a file server). All of the 
laboratory computers are connected to the Internet 
which allows users to efficiently download geospatial 
data from inside or outside the university and commu- 
nicate with one another. All the computers have access 
to high-quality input (e.g., coordinate digitizing table, 
scanner) and output devices (e.g., E-size plotter, color 


laser printer). Each computer has access to >5 TB of 
mass storage and the ability to back up important files 
locally or at a common repository. Whiteboards are 
present to list tasks and promote discussion among the 
analysts. The digital overhead projector allows the in- 
structor to demonstrate concepts and interactively per- 
form image processing. It is also possible to perform 
digital image processing over the Internet and access 
data in a “cloud.” 


Interactive Graphical User Interface 


One of the best scientific visualization environments 
for the analysis of remote sensor data takes place when 
the analyst communicates with the digital image pro- 
cessing system interactively using a point-and-click 
graphical user interface (GUI). Most sophisticated 1m- 
age processing systems are configured with a friendly, 
point-and-click GUI that allows rapid display of imag- 
es and the selection of important image processing 
functions. Easy to use digital image processing graphi- 
cal user interfaces are found in: 


¢ Exelis Inc., Environment for Visualizing Images 
(ENVI®) (Figure 3-6), 
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ENVI Graphical User Interface (GUI) 
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FIGURE 3-6 The Exelis, Inc., Environment for Visualizing Images (ENVI®) graphical user interface. The “Build 3D Cube” pro- 
gram is being used to create a 3D datacube of AVIRIS hyperspectral imagery of wetland and salt evaporation ponds near 
Moffett Field, CA. There are 300 rows, 500 columns, and 56 of 224 AVIRIS bands in this particular dataset. Bands 25 (center at 
0.6381 ym; red), 16 (0.5488 ym; green), and 7 (0.4603 pm; blue) are displayed on top of the cube in a natural color composite 
(interface courtesy of Exelis, Inc.; AVIRIS data courtesy of NASA Jet Propulsion Laboratory). 


¢ Intergraph, Inc., ERDAS IMAGINE® and Leica 
Photogrammetry Suite (LPS®) (Figure 3-7), 


¢ Clark University’s IDRISI®, 
¢ Purdue University’s MultiSpec®, and others. 


Several geographic information systems perform digi- 
tal image processing and use effective GUIs, including: 


¢ Esri, Inc., ArcGIS® image Classification, Feature 
Analyst® and Spatial Analyst® interfaces (Figure 
3-8), and 

¢ GRASS GIS®. 


Adobe, Inc., Photoshop® has a time-tested GUI used 
by thousands of people each day. The software is espe- 
cially useful for enhancing photographs and images 
that have three or fewer bands of data and geometric 
rectification and classification are not required. 


Batch Processing 


Non-interactive, batch processing can be of value for 
time-consuming processes such as image rectification, 


segmentation, mosaicking, orthophoto generation, 
and special filtering. Batch processing frees up labora- 
tory PCs or workstations during peak demand because 
the time-consuming jobs can be put in a que and exe- 
cuted when the computer is otherwise idle (e.g., during 
early morning hours). Batch processing can also be 
useful during peak hours because it allows the analyst 
to set up a series of operations that can be executed in 
sequence without operator intervention. 


Computer Operating System 
So. and Compiler(s) 


The operating system is the first program loaded into 
memory (RAM) when the computer is turned on. The 
operating system controls all of the computer’s higher 
order functions and resides in RAM at all times. The 
operating system provides the user interface, controls 
multitasking, handles input and output to the hard 
disk and all peripheral devices such as DVDs, scanners, 
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FIGURE 3-7 The Intergraph ERDAS Imagine® graphical user interface with the “Multispectral” pallet highlighted. The 
“sharpness” of this image near Beaufort, SC, is in the process of being adjusted (user interface courtesy of Intergraph, Inc., 
part of Hexagon, Inc.; image courtesy of Dan Morgan, Beaufort County GIS Department). 


printers, plotters, and the color display. All digital im- 
age processing software must communicate with the 
operating system. 


Different types of digital image processing software are 
able to run on most major operating systems (e.g., 
Windows, UNIX, Linux, Mac OS; Tables 3-1 and 3-5). 
Users should select digital image processing software 
that runs on an operating system that they are com- 
fortable with. This helps reduce the learning curve as 
you begin working with the image processing software. 
If you are going to run the applications software on 
your personal computer, be aware that you may have to 
upgrade to a new or different operating system in order 
to run the image processing software. If you are install- 
ing the image processing software on an existing com- 
puter you must also be sure that the computer has 
sufficiently fast CPU(s), enough memory (RAM), and 


adequate hard disk space for the image processing soft- 
ware to run efficiently. 


As discussed in Chapter 2, it is important to have a 
quality scanner to digitize hard-copy imagery such as 
panchromatic, natural color, and/or color-infrared aer- 
ial photography. Aerial photographs are typically 9 x 9 
in. in size. Therefore, it is important to have a scanner 
with an effective area of at least 12 x 16 in. so that an 
entire 9 X 9 in. aerial photograph can be scanned in 
one pass (Figure 3-4). 


A GIS should be able to output high-quality maps, im- 
ages, charts, and diagrams in both small (e.g., A-size 
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FIGURE 3-8 The Esri ArcGIS ArcMap® graphical user interface with the image Classification, Feature Analyst®, and Spatial 
Analyst® extensions turned on. The classification user interface is being used to collect training samples of water and wet- 
land adjacent to Beaufort, SC (user interface courtesy of Esri, Inc.; image courtesy of Dan Morgan, Beaufort County GIS 


Department). 


plots 8.5 x 11 in.) or large formats (e.g., E-size plots 36 
x 48 in.). To accomplish this, small- and large-format 
printers are required. Inexpensive ink-jet or color laser 
printers can be used for small format printing, and E- 
sized ink-jet plotters can be used for large formats. 


Data Storage and Archiving 
Considerations 


Digital image processing of remote sensing and related 
GIS data requires substantial mass storage resources. 
Therefore, the mass storage media should allow rela- 
tively rapid access times, have longevity (i.e., last for a 
long time), and be inexpensive. 


The best way to make digital remote sensor data rapid- 
ly available to CPU(s) is to place the data on a hard 
disk, compact disk (CD), digital video disk (DVD), 
flash drive, and/or in the Internet “cloud” where each 
pixel of the data matrix may be accessed at random 
(not serially) and at great speed (i.e., within microsec- 
onds). Because of the reduced cost of mass storage, it is 
common for digital image processing laboratories to 
have gigabytes of hard-disk mass storage associated 
with each PC or workstation. For example, each PC in 
the laboratory shown in Figure 3-5 has access to > 1 
TB of hard disk mass storage. Some image processing 
laboratories use RAID (redundant arrays of inexpen- 
sive hard disks) technology in which two or more 
drives working together provide increased performance 
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and various levels of error recovery and fault tolerance. 
It is now common for scientists and students to use rel- 
atively inexpensive CDs, DVDs, and especially flash 
drives to back up their remote sensing data and results 
after every session. The cost of hard disk, CD, DVD, 
flash drive, and cloud data storage and access per giga- 
byte continues to decline dramatically, which is won- 
derful for remote sensing science and education. 


Cloud computing is becoming a technology that almost 
every industry that provides or uses hardware, soft- 
ware, and data storage will utilize (Kouyoumjia, 2010; 
Yang and Huang, 2013). Cloud computing may even- 
tually lead to a future where we do not compute on lo- 
cal computers. Rather, computing will occur on 
centralized facilities operated by third-party computer 
and data storage utilities. In a cloud computing envi- 
ronment, the data or software stored on remote servers 
is provided via the Internet to all types of computers. 


Presently, there are two main types of cloud comput- 
ing: public and private. In the public cloud, the infra- 
structure and services are owned and sold by a separate 
organization. Data space on the public cloud is avail- 
able to all computer users. Private cloud computing is 
available for companies or organizations that are not 
comfortable placing their data files in a public cloud. A 
private cloud provides data storage in a remote loca- 
tion that is maintained behind a restrictive firewall. 


Archiving Considerations—Longevity 


Significant sums of money are spent purchasing re- 
mote sensor data by commercial companies, natural 
resource agencies, and universities. Storing massive 
amounts of original remote sensor data, processed re- 
mote sensor data, and project results is no trivial mat- 
ter. Unfortunately, most of the time not enough 
attention is given to how the expensive data are ar- 
chived to protect the long-term investment. Figure 3-9 
depicts several types of analog and digital remote sen- 
sor data mass storage devices and the average time to 
physical obsolescence, that is, when the media begin to 
deteriorate and/or information is lost. Interestingly, 
properly exposed, washed, and fixed analog black-and- 
white aerial photography negatives have considerable 
longevity, often more than 100 years. That is why we 
still have pictures of Abraham Lincoln from the 1860s. 
Color negatives with their respective dye layers have 
longevity, but not as much as the black-and-white neg- 
atives. Black-and-white paper prints have greater lon- 
gevity than color prints because the dyes in the color 
prints often begin to fade after a few years. 


For all practical purposes, floppy disk and magnetic 
tape media are obsolete (Figure 3-9). The magnetic 
tape medium is particularly problematic because it 


stretches over time. The dimensional instability can 
make it very difficult to read the data stored on tape. 
Interestingly, there are thousands of remote sensing 
images acquired by SPOT, Landsat, and other sensors 
during the 1970s through 1990s stored on tape in labo- 
ratories throughout the world that need to be trans- 
ferred to more robust storage media (e.g., DVD) before 
it is too late. 


Hard disks, CDs, DVDs, and even flash drives provide 
long-term storage potential (>100 years) (Figure 3-9). 
However, it is very important to remember when ar- 
chiving digital remote sensor data that sometimes it is 
the loss of the read—write software and/or the read— 
write hardware (the drive mechanism and heads) that 
becomes a problem and not the digital media itself 
(Rothenberg, 1995). Therefore, as new computers are 
purchased it is a good idea to archive all the computer 
system hardware and software that is representative of 
a certain computer era so that one can always read any 
data stored on old digital mass storage media. This in- 
cludes the computer, hard-disk, monitor, keyboard, 
mouse, DVD, etc. 


While cloud storage will continue to grow and evolve, 
there are some limitations. The data transfer rate be- 
tween an individual computer and the cloud can be 
slow. This is a problem when large geospatial datasets 
must be accessed via the cloud. Privacy and security 
are two other concerns with cloud-computing. In some 
cases, customer or government data security issues 
may preclude storing sensitive data in the cloud (Kouy- 
oumjia, 2010). Finally, cloud data storage can be a via- 
ble long-term data storage technology if the 
commercial firms managing the cloud remain in busi- 
ness and rigorously maintain the data through time. 


Computer Display Spatial 
and Color Resolution 


The display of remote sensor data on a computer 
screen is one of the most fundamental tasks of digital 
image analysis. Careful selection of the computer dis- 
play characteristics will provide the optimum visual im- 
age analysis environment for the human interpreter. 
The two most important characteristics are computer 
display spatial and color resolution. 


Computer Screen Display Resolution 


The digital image processing system should be able to 
display at least 1,024 rows by 1,024 columns on the 
computer screen at one time. This allows larger geo- 
graphic areas to be examined and places the terrain of 
interest in its regional context. Most Earth scientists 
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Remote Sensor Data Storage Media and their Longevity 
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FIGURE 3-9 Different types of analog and digital remote sensor data mass storage alternatives and the average time to 
physical obsolescence. The loss of software or hardware to read the archived digital media is often the most serious problem. 


prefer this regional perspective when performing ter- _—_ single 1,024 x 1,024 display provides the information 
rain analysis using remote sensor data. It is disconcert- at a glance. An ideal screen display resolution is 1,600 
ing to have to analyze four 512 x 512 images whena = X 1,200 pixels. 


CHAPTER 3 


Computer Screen Color Resolution 


The computer screen color resolution is the number of 
gray-scale tones or colors (e.g., 256) that can be dis- 
played on the computer screen at one time out of a pal- 
ette of available colors (e.g., 16.7 million). For many 
applications, such as high-contrast black-and-white 
linework cartography, only 1 bit of color is required 
(i.e., either the line is black or white [0 or I]). For more 
sophisticated computer graphics for which many 
shades of gray or color combinations are needed, up to 
8 bits (or 256 colors) may be required. Most thematic 
mapping and GIS applications may be performed well 
by systems that display just 64 user-selectable colors 
out of a palette of 256 colors. 


Conversely, the analysis and display of remote sensor 
image data require much higher screen color resolution 
than cartographic and GIS applications. For example, 
most relatively sophisticated digital image processing 
systems can display a tremendous number of unique 
colors from a large color palette (e.g., 16.7 million). 
The primary reason for these color requirements 1s that 
image analysts must often display a composite of sev- 
eral images at one time on the screen. This process is 
called color compositing. For example, to display a typ- 
ical color-infrared image of Landsat Thematic Mapper 
data, it is necessary to composite three separate 8-bit 
images (e.g., the green band [TM 2 = 0.52 to 0.60 um], 
the red band [TM 3 = 0.63 to 0.69 uml], and the reflec- 
tive infrared band [TM 4 = 0.76 to 0.90 um]). To ob- 
tain a true-color composite image that provides every 
possible color combination for the three 8-bit images 
requires that 274 colors (16,777,216) be available in the 
palette (Table 3-3). Such true-color, direct-definition 
systems are relatively expensive because every pixel lo- 
cation must be bit-mapped. This means that there must 
be a specific location in memory that keeps track of the 
exact blue, green, and red color value for every pixel. 
This requires substantial amounts of computer memo- 
ry which are usually present in what is called an image 
processor (discussed in the next chapter). Given the 
availability of image processor memory, the question 
is: what is adequate color resolution? 


Generally, 4,096 carefully selected colors out of a very 
large palette (e.g., 16.7 million) seem to be the mini- 
mum acceptable for the creation of remote sensing col- 
or composites. This provides 12 bits of color, with 4 
bits available for each of the blue, green, and red image 
planes (Table 3-3). For image processing applications 
other than compositing (e.g., black-and-white image 
display, color density slicing, pattern recognition clas- 
sification), the 4,096 available colors and large color 
palette are adequate. However, the larger the palette 
and the greater the number of displayable colors at one 
time, the better the representation of the remote sensor 
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TABLE 3-3 Image processing memory required to produce 
various numbers of displayable colors on the computer 
screen. 


Maximum Number of 
Image Processor Colors Displayable at 
Memory (bits) One Time on the Screen 
1 2 (black and white) 
2 4 
3 8 
4 16 
5 32 
6 64 
7 128 
8 256 
9 512 
10 1,024 
11 2,048 
|2 4,096 
118) 8,192 
14 16,384 
15 32,768 
16 65,536 
IZ 131,072 
18 262,144 
24 OW 2N6 


data on the screen for visual analysis. More informa- 
tion about how images are displayed using an image 
processor is in Chapter 5. The network configured in 
Figure 3-4 has six 24-bit color workstations. 


Several remote sensing systems now collect data with 
10-, 11-, and even 12-bit radiometric resolution with 
brightness values ranging from 0 to 1023, 0 to 2047, 
and 0 to 4095, respectively. Unfortunately, despite ad- 
vances in video technology, at the present time it 1s nec- 
essary to generalize (i.e., dumb down) the radiometric 
precision of the remote sensor data to 8 bits per pixel 
for display on the computer screen simply because cur- 
rent video display technology cannot handle the de- 
mands of the increased precision. 


g Digital Image Processing 
Software Considerations 


The digital image processing software selected is criti- 
cal to the successful completion of the image process- 
ing project or research. A good rule of thumb is to 
always use digital image processing software that has 
an excellent reputation. You may be making decisions 
based on the output from the image processing that 
can impact people, flora, and fauna. Your reputation 
will be dependent upon how carefully you structure the 
research question or application and the proper use of 
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algorithms and procedures within the image processing 
software. 


Image Processing Functions 


Many of the most important functions performed us- 
ing digital image processing systems are summarized in 
Table 3-4. Personal computers now have the comput- 
ing power to perform each of these functions. This 
textbook examines many of these digital image pro- 
cessing functions in subsequent chapters, including ra- 
diometric and geometric preprocessing (Chapters 6 
and 7), enhancement (Chapter 8), information extrac- 
tion (Chapters 9 through 11), change detection (Chap- 
ter 12), and accuracy assessment (Chapter 13). 


It is not good for remotely sensed data to be analyzed 
in a vacuum. Remote sensing information fulfills its 
promise best when used in conjunction with other an- 
cillary data (e.g., soils, elevation, and slope) often 
stored in a geographic information system (GIS). 
Therefore, the ideal digital image processing system 
should be able to process the digital remote sensor data 
as well as perform any necessary GIS processing (du 
Plessis, 2012). It is usually not efficient to exit the digi- 
tal image processing system, log into a GIS system, 
perform a required GIS function, and then take the 
output of the procedure back into the digital image 
processing system for further analysis. Most integrated 
systems perform both digital image processing and 
GIS functions and consider map data as image data 
(or vice versa) and operate on them accordingly. 


Most digital image processing systems have some limi- 
tations. For example, most systems can perform multi- 
spectral classification on a few bands of imagery, but 
only a few systems can perform hyperspectral analysis 
on hundreds of bands of imagery. Similarly, only a few 
systems can perform soft-copy photogrammetric oper- 
ations on overlapping stereoscopic imagery displayed 
on the screen and generate digital orthophotographs or 
digital elevation models. Also, only a few digital image 
processing systems incorporate expert systems or neu- 
ral networks or fuzzy logic. Finally, systems of the fu- 
ture should provide detailed image lineage (genealogy) 
information about the processing applied to each im- 
age. The image lineage information (metadata) is indis- 
pensable when the products derived from the analysis 
of remotely sensed data are subjected to intense scruti- 
ny as in environmental litigation. 


Digital Image Processing Software 


Some commonly-used digital image processing pro- 
grams are listed in Table 3-5. The information provided 
in the table does not endorse any particular digital im- 
age processing software program. There is no “one- 


TABLE 3-4 Image processing functions in quality digital im- 
age processing systems. 


Preprocessing 

o Radiometric correction of error introduced by the sensor sys- 
tem electronics/optics and/or environmental effects (includes 
relative image-to-image normalization and absolute radio- 
metric correction of atmospheric attenuation) 

o Geometric correction (image-to-image registration or image- 
to-map rectification) 


Display and Enhancement 

o Black-and-white computer display (8-bit) 

o Color composite computer display (24-bit) 

o Magnification (zooming), reduction, roaming 

o Contrast manipulation (linear, nonlinear) 

o Color space transformations (e.g., RGB to IHS) 

o Image algebra (e.g., band ratioing, image differencing, NDVI, 
SAVI, Kauth-Thomas, EVI) 

o Spatial filtering (e.g., low-pass, high-pass, bandpass) 

o Edge enhancement (e.g., Kirsch, Laplacian, Sobel) 

0 Principal components analysis 

o Texture transforms 

o Frequency transformations (e.g., Fourier, Walsh) 

o Digital elevation models (e.g., creation of TIN, interpolation 
via inverse distance weighting, or kriging, analytical hill shad- 
ing, calculation of slope, aspect) 

o Three-dimensional transformations (e.g., image draping over 
digital elevation models) 

o Image animation (e.g., movies, change detection) 


Information Extraction 

o Pixel brightness value (BV;,,) 

o Black-and-white or color density slice 

o Transects (spatial and spectral) 

o Univariate and multivariate statistical analysis 

o Feature (band) selection (graphical and statistical) 

o Supervised (e.g., minimum distance, maximum likelihood) and 
unsupervised classification (e.g., ISODATA) 

o Geographic object-based image analysis (GEOBIA) 

o Incorporation of ancillary data during classification 

o Expert system image analysis including rule-based decision- 
tree classifiers and machine learning 

o Neural network analysis and use of support vector machines 

o Fuzzy logic classification 

o Hyperspectral data analysis 

o LiDAR data analysis 

o RADAR data analysis 

o Photogrammetric mapping (creation of orthoimages, extrac- 
tion of digital elevation models and planimetric detail) 

o Change detection 

o Accuracy assessment (descriptive and analytical) 


Image and Map Cartographic Composition 
o Scaled Postscript level Ill output of images and maps 


Integrated Geographic Information Systems 
o Raster (image)-based GIS 
o Vector (polygon)-based GIS (must allow polygon overlay) 


Utilities 

o Network (e.g., local area network, Internet) 
o Image compression (single image, video) 

o Import and export of various file formats 


Metadata and Image/Map Lineage Documentation 
o Metadata 
o Complete image and GIS file processing history 
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size-fits-all” digital image processing software program 
that everyone should use. Before purchasing digital im- 
age processing software, analysts should carefully eval- 
uate the functions and capabilities to see if they match 
current and potential image analysis requirements. In 
most cases, representatives from the digital image pro- 
cessing software companies will be pleased to demon- 
strate the capability of the software. Analysts should 
ask software representatives to demonstrate specific 
types of geospatial analysis that are of special interest. 


Characteristics of the most commonly-used digital im- 
age processing programs are summarized in Table 3-5. 
Most of the software uses the Windows operating sys- 
tem. A robust remote sensing-oriented digital image 
processing system should be able to perform: 


¢ radiometric and geometric correction; 


* image enhancement (e.g., filtering, vegetation indi- 
ces, principal components analysis); 


* traditional image classification (e.g., unsupervised 
clustering, supervised maximum likelihood classifi- 
cation); 


¢ decision-tree classification, neural network analysis, 
and machine-learning; 


* geographic object-based image analysis (GEOBIA) 
(Blaschke et al., 2014); 


* hyperspectral image analysis; 

¢ LiDAR data analysis; 

¢ RADAR image analysis; 

¢ photogrammetric mapping; and 
¢ change detection. 


Basically, there are four types of digital image process- 
ing programs: 


¢ those used to analyze any type of image (e.g., medi- 
cal x-rays, terrestrial photographs, personal photo- 
graphs) such as Adobe Photoshop. Most of these 
cannot perform atmospheric or geometric correc- 
tion nor can they perform traditional supervised 
and/or unsupervised image classification. MAT- 
LAB, on the other hand, has a robust image pro- 
cessing toolbox that can be used to perform a 
variety of image enhancements, perform image res- 
toration, image segmentation, and can geometri- 
cally rectify imagery (McAndrew, 2004; Gonzalez et 
al., 2009; MathWorks, 2014). 


* those designed specifically for analyzing digital 
remote sensor data such as ERDAS Imagine, ENVI, 
PC Geomatica, IDRISI, eCognition, Feature Ana- 
lyst, LIDAR Analyst, LP360, SOCET, TNTmips, 
VIPER; 


* sophisticated GIS software that has digital image 
processing capabilities such as ArcGIS and GRASS 


DIGITAL IMAGE PROCESSING HARDWARE AND SOFTWARE 127 


(Geographic Resources Analysis Support System); 
and 


* computer-aided design (CAD) software with some 
image processing capability such as AUTOCAD. 


Below are some observations about multispectral, hy- 
perspectral, LIDAR, RADAR, photogrammetric map- 
ping, change detection, and integrated digital image 
processing/GIS software. The observations relate to 
the information found in Table 3-5. 


Multispectral Digital Image Processing 
Software 

ERDAS Imagine, ENVI, PCI Geomatica, TNTmips, 
and IDRISI are heavily adopted digital image process- 
ing systems used for analysis of aircraft and satellite 
multispectral remote sensor data. All have radiometric 
and geometric preprocessing algorithms, a variety of 
image enhancement and analysis routines, and useful 
change detection modules. Some can process RADAR 
imagery. 


Geographic Object-based Image Analysis 
(GEOBIA) 

eCognition, Feature Analyst, IDRISI, and ENVI have 
excellent geographic object-based image analysis (GE- 
OBIA) programs. ERDAS and MATLAB also have 
image segmentation and processing capabilities. 


Hyperspectral Digital Image Processing 
Software 

ENVI and VIPER are rigorous hyperspectral image 
analysis programs. Both have sophisticated radiomet- 
ric (atmospheric) correction capabilities and a diversity 
of algorithms to extract information from hyperspec- 
tral data. ENVI is a commercial product while VIPER 
is freeware available from the University of California 
at Santa Barbara. ACORN (Atmospheric Correction 
Now) is a commercial atmospheric correction pro- 
gram. ERDAS and IDRISI also have useful hyperspec- 
tral analysis capability. MultiSpec is a freeware 
program available from Purdue University with hyper- 
spectral image processing functions. 


LiDAR Digital Image Processing Software 
SOCET GXP software has extensive enterprise level 
LiDAR 3D processing capability used primarily by in- 
telligence gathering agencies (O’Neil-Dunne, 2012). Li- 
DAR Analyst and QCoherent LP360 are sophisticated 
LiDAR 3D information extraction programs used ex- 
tensively for civil engineering, intelligence-gathering 
and forestry applications. ArcGIS, GRASS, ERDAS, 
IDRISI, TNTmips, ER Mapper, and ENVI can create 
triangular-irregular network (TIN) or raster digital ter- 
rain models from LiDAR point cloud data (i.e., 
masspoints). 
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TABLE 3-5 Selected digital image processing software programs and their characteristics. The greater the capability, the 


greater the number of bullets eeeee. The Internet address of the vendor or agency is provided. 


Operating | Preprocessing Information Extraction 
System = : ae 
Digital Image (Windows Rectifi-| Atmos- |Enhance-| Tradi- | Decision Hyper- Photo- 
Processing Software| Mac Unix) cation| pheric | ment |tional| Tree GOBIA |spectral |LiDAR| RADAR | grammetric 
ACORN® Windows eocoe 
imspec.com 
ArcGIS® ArcMap® | Windows/ | eeee ecco ecco 
esri.com UNIX 
AUTOCAD® Windows/ | eeeee eo ec 
autodesk.com Mac 
TNTmips® Windows | ececeee ecco ecccoe | ceccce eocce ee ecco ee eo ee 
microimages.com 
eCognition® Windows | eeceee | eccce eocce eco eocce eecce eocce ee ee e 
ecognition.com 
ENVI® Windows/ | eececee | ecocce ecccoe | eccce eocce rryyy) eecce ecce ecco eco 
exelisvis.com UNIX 
ERDAS ER Mapper® Windows/ | eececee | eocce eocce | eccce eocce eecce eevee eco ecce 
intergraph.com UNIX 
ERDAS Imagine® Windows/ | eececee | eccce ecccoe | eccce eocce rryyy) eecce eco eccce eco 
intergraph.com UNIX 
Feature Analyst® Windows | eeceee eocce eocce 
overwatch.com 
GRASS GIS® Windows/ | eeeee eco eccoccee | cocce 
grass.osgeo.org Mac/UNIX 
IDRISI® Windows/ | eececee | eccce eccce | ceccce eccce eco eco ee eco 
clarklabs.org UNIX 
Leica Photogram- Windows/ | eeeee eo eocce éaese 
metry Suite (LPS)® UNIX 
intergraph.com 
LIDAR Analyst® Windows | ***** eocce 
overwatch.com 
LP360® Windows | eeeee ° ecce eccce 
qcoherent.com 
MATLAB® Windows/ | eeeee ° ecccee | cocce ee eocce 
Mathworks.com Mac/Linux 
MrSid® Windows/ image compression 
lizardtech.com Linux 
MultiSpecO® Windows/ ° ° coe | cccce ecco 
engineering. pur- Mac 
due.edu/~biehl/Mul- 
tiSpec/ 
PCI Geomatica® Windows/ | eeceee | eccce eocce | cccce | ecocce eooce eocee |ecccee| cocce ecco 
pcigeomatics.com UNIX 
Photoshop® Windows/ eccce 
adobe.com Mac 
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TABLE 3-5 (continued) Selected digital image processing software programs and their characteristics. The greater the capa- 


bility, the greater the number of bullets eeeee. 


Operating | Preprocessing Information Extraction 
System ee e ane 

Digital Image (Windows Rectifi-) Atmos- /Enhance-| Tradi- | Decision Hyper- Photo- 
Processing Software| Mac Unix) cation| pheric | ment | tional] Tree OBIA  |spectral |LiDAR| RADAR | grammetric 
R® Windows/ eocece ecco eccce eco 
www.r-project.org Mac 
SOCET GXP® Windows/ | eeeee | eccce eocee | eccce eocece eocce eocee |cec0ee| cocce eocce 
socetgxp.com UNIX/Linux 
VIPER® Windows | eececee | eocee eo eo eooce 
vipertool.org 


RADAR Digital Image Processing Software 
Only a few of the commercially-available digital image 
processing programs can process RADAR (single po- 
larization) or Polarimetric RADAR (multiple polariza- 
tion) data. ERDAS Imagine, ENVI, PCI Geomatica, 
and IDRISI have RADAR processing modules. 


Photogrammetric Mapping Software 
Photogrammetric mapping involves the processing of 
stereoscopic digital aerial photography to extract pla- 
nimetric information (e.g., building footprints, road 
centerlines, fences, tree footprints, drainage networks) 
and/or three-dimensional topographic information. 
SOCET software by BAE Systems, Inc., is a quality 
soft-copy photogrammetry program. However, few 
outside of intelligence-gathering agencies have access 
to its capabilities (O’Neil-Dunne, 2012). Intergraph’s 
Leica Photogrammetry Suite (LPS) has a user-friendly 
interface and extensive photogrammetric mapping ca- 
pabilities. ERDAS, ENVI, IDRISI, TNTmips, and ER 
Mapper have some soft-copy photogrammetric pro- 
cessing functions, most notably those necessary to cre- 
ate orthophotography, and digital terrain models. 


Change Detection 

Some software such as ERDAS (e.g., DeltaQue), ID- 
RISI (e.g., Land Change Modeler for ArcGIS), ENVI 
(e.g., SPEAR), and Esri (e.g., Change Matters; Green, 
2011) have change detection programs or wizards for 
analyzing multiple date aircraft and/or satellite data 
(Jensen et al., 2012). Pictometry Analytics (2014) mar- 
kets ChangeFindr software, which works with both 
vertical and oblique digital aerial photography. 


Integration of Digital Image Processing and 
GIS Functions 

ArcGIS is the most sophisticated vector-based geo- 
graphic information system. Remote sensing data and 
remote sensing-derived thematic information is easily 
incorporated and analyzed in ArcGIS ArcMap. IDRI- 
SI and GRASS are powerful raster-based geographic 


information systems that easily incorporate vector in- 
formation for GIS modeling. ERDAS and ENVI can 
incorporate vector information in their various GIS 
analysis operations. ERDAS Apollo is a comprehen- 
sive digital image processing, photogrammetry, and 
GIS program (du Plessis, 2012). TNTmips is an inte- 
grated GIS, image processing, CAD, desktop cartogra- 
phy, and geospatial database management program. 


Cost 


Companies, public agencies, and academic institutions 
have limited financial resources that must be used care- 
fully. Therefore, the cost of commercial digital image 
processing software is a serious consideration. Digital 
image processing software is expensive. This may be 
the result of the still closed-source nature of most digi- 
tal image processing software. The price for a single 
digital image processing software license is highest for 
commercial users, somewhat lower for public agencies, 
and much lower for academic institutions. 


Open-Source Digital Image Processing 
Software 

If software cost is a concern, open-source digital image 
processing software, such as GRASS or MultiSpec, 
may be the best solution. 


Open-Source Statistical Analysis Software that 
can be used for Digital Image Processing 

In addition to commercial SAS and SPSS statistical 
analysis software packages, advanced digital image 
processing can be performed using R—an open-source 
programming language available at http://www.r-proj- 
ect.org/. R is widely used by statisticians and data min- 
ers to develop statistical analysis software. The types of 
processes that can be applied to remote sensing images 
are available in various user-submitted R packages 
(Venables et al., 2012). Image analysts often use pack- 
ages from the Machine Learning & Statistical Learning 
group to perform neural network analysis, decision- 
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tree analysis, CART, random forests, C5.0, support 
vector machine (SVM) analysis, and others discussed 
in Chapter 10. 


Digital Image Processing and 
the National Spatial Data 
Infrastructure 


Laypersons and scientists who use remote sensing data 
or share products derived from remotely sensed data 
should be aware of very specific spatial data standards 
developed by the Federal Geographic Data Committee 
(FGDC). The FGDC is an interagency committee of 
representatives from the Executive Office of the Presi- 
dent, the Cabinet, and independent agencies. The 
FGDC is developing the National Spatial Data Infra- 
structure (NSDI) in cooperation with organizations 
from state, local and tribal governments, academics, 
and the private sector. The NSDI encompasses poli- 
cies, standards, and procedures for organizations to co- 
operatively produce and share geographic data 
(FGDC NSDI, 2014). 


The American National Standards Institute’s (ANSI) 
Spatial Data Transfer Standard (SDTS) is a mecha- 
nism for archiving and transferring spatial data (in- 
cluding metadata) between dissimilar computer 
systems. The SDTS specifies exchange constructs, such 
as format, structure, and content, for spatially refer- 
enced vector and raster (including gridded) data. Actu- 
al use of SDTS to transfer spatial data is carried out 
through its profiles. The FGDC Raster Profile stan- 
dard is of particular interest because it provides specifi- 
cations for transferring spatial datasets in which 
features or images are represented in raster or gridded 
form, such as digital elevation models, digital ortho- 
photo quarter quads (DOQQ), and digital satellite im- 
agery (FGDC SDTS, 2014). 
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4 IMAGE QUALITY ASSESSMENT 


AND STATISTICAL EVALUATI 
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Many remote sensing datasets contain high-quality, ac- 
curate data. Unfortunately, sometimes error (or noise) 
is introduced into the remote sensor data by a) the en- 
vironment (e.g., atmospheric scattering), b) random or 
systematic malfunction of the remote sensing system 
(e.g., an uncalibrated detector creates striping), or c) 
improper airborne or ground processing of the remote 
sensor data prior to actual data analysis (e.g., an inac- 
curate analog-to-digital conversion). Therefore, the 
person responsible for analyzing the digital remote 
sensor data should first assess its quality and statistical 
characteristics. This is normally accomplished by: 


¢ looking at the frequency of occurrence of individual 
brightness values in the image displayed in histo- 
gram format, 


* viewing on a computer monitor individual pixel 
brightness values at specific locations or within a 
geographic area, 

* computing fundamental univariate descriptive sta- 
tistics to determine if there are unusual anomalies in 
the image data, and 


* computing multivariate statistics to determine the 
amount of between-band correlation (e.g., to iden- 
tify redundancy). 


A Overview 


This chapter first reviews basic elements of statistical 
sampling theory. It then introduces the histogram and 
its significance to digital image processing of remote 
sensor data. Various methods of viewing individual 
pixel values and geographic areas of individual pixel 
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values are then presented. Algorithms for the compu- 
tation of univariate and multivariate statistics are in- 
troduced, including the identification of the minimum 
and maximum value for each band of imagery, the 
range, mean, standard deviation, and between-band 
covariance and correlation. Finally, geostatistical anal- 
ysis is introduced that can be of value for obtaining in- 
formation about spatial autocorrelation in imagery 
and when performing spatial interpolation. 


\, Image Processing 
Mathematical Notation 


The following notation is used to describe the mathe- 
matical operations applied to the digital remote sensor 
data: 

i =a row (or line) in the imagery 

j =acolumn (or sample) in the imagery 

k =a band of imagery 


/ = another band of imagery 


n = total number of picture elements 
(pixels) in an array 


BV = brightness value at row 7, column j, and 
band k 


BV; = ith brightness value in band k 


BV;,; = ith brightness value in band / 
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min, = minimum brightness value of band k 
max; = maximum brightness value of band k 
range, = range of brightness values in band k 


quant; = quantization level of band k (e.g., = 
0 to 255; 2!* = 0 to 4,095) 


uu, = mean of band k 
var), = variance of band k 
sj = standard deviation of band k 
skewness; = skewness of a band k distribution 
kurtosis; = kurtosis of a band k distribution 


covy; = covariance between pixel values in two 
bands, k and / 


ryy = correlation between pixel values in two 
bands, A and / 


X,, = measurement vector for class c composed 
of brightness values (BV;;,) from row i, 
column j, and band k 


M,.= mean vector for class c 
M,= mean vector for class d 
X.~ = mean value of the data in class c, band k 


So¢ = Standard deviation of the data in class c, 
band k 


Vad = covariance matrix of class c for bands k 
through /; shown as V,, 


Vaki = covariance matrix of class d for bands k 
through /; shown as V 


( Sampling Theory 


Digital image processing is performed on only a sam- 
ple of all available remote sensing information. There- 
fore, it is useful to review several fundamental aspects 
of elementary statistical sampling theory. A population 
is an infinite or finite set of elements. An infinite popu- 
lation would be all possible images that might be ac- 
quired of the entire Earth in 2013. All Pleiades images 


of Charleston, SC, obtained in 2013 would be a finite 
population. 


Types of Sampling 


A sample is a subset of the elements taken from a pop- 
ulation used to make inferences about certain charac- 
teristics of the population. For example, we might 
decide to analyze a November 1, 2013, Pleiades image 
of Charleston, SC. If observations with certain charac- 
teristics are systematically excluded from the sample ei- 
ther deliberately or inadvertently (such as selecting 
images obtained only in the fall of the year), it is a bi- 
ased sample. The difference between the true value of a 
population characteristic and the value of that charac- 
teristic inferred from a sample is the sampling error. 


The goal of sampling is to collect an unbiased repre- 
sentative sample of the population (Jensen and Shum- 
way, 2010). A random sample with replacement occurs 
when every observation has an equal chance of being 
selected. This is one of the most simple sampling 
schemes, and it usually ensures that there is no bias in 
the sample. An example of a simple random sample 
used to assess the accuracy of a remote sensing-derived 
thematic map is presented in Chapter 13. 


A systematic sample can be implemented according to 
a predetermined system (e.g., collect data every 100 m 
in the x- and y-directions). In effect, most remote sens- 
ing data are collected systematically (e.g., Landsat 8 
systematically collects panchromatic imagery every 15 
X 15 m). An example of a systematic sample used to 
assess the accuracy of a remote sensing-derived the- 
matic map is presented in Chapter 13. 


In some cases, a stratified sample is most appropriate. 
A stratified random sample could be implemented when 
the analyst knows that the study area contains differ- 
ent sub-populations and he or she makes an effort to 
sample within each sub-population (i.e., strata). This 
helps ensure that all of the variation present in the spa- 
tial dataset is accounted for (Jensen and Shumway, 
2010). An example of a stratified random sample used 
to assess the accuracy of a remote sensing-derived the- 
matic map is presented in Chapter 13. 


Large samples drawn randomly from natural popula- 
tions usually produce a symmetrical frequency distri- 
bution such as that shown in Figure 4-la. Most values 
are clustered around a central value, and the frequency 
of occurrence declines away from this central point. A 
graph of the distribution appears bell-shaped and is 
called a normal distribution. Many statistical tests used 
in the analysis of remotely sensed data assume that the 
reflectance, emittance, or back-scattered values record- 
ed in a scene are normally distributed. Unfortunately, 
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remotely sensed data may not be normally distributed, 
and the analyst must be careful to identify such condi- 
tions. In such instances, nonparametric statistical theo- 
ry may be preferred. 


\, The Histogram and its 
J Significance to Digital Image 
Processing 


The histogram is a useful graphic representation of the 
information content of a single band of remotely 
sensed data (Papp, 2010). Histograms for each band of 
imagery are often displayed and analyzed in many re- 
mote sensing investigations because they provide the 
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FIGURE 4-1 Relative position of measures 
of central tendency for commonly encoun- 
tered frequency distributions. a) Normally 
distributed data. b) Bi-modal distribution. c) 
Uniform distribution. d) Negatively skewed 
distribution. e) Positively skewed distribu- 
tion. 


analyst with an appreciation of the quality of the origi- 
nal data (e.g., whether it is low in contrast, high in con- 
trast, or multimodal in nature). In fact, many analysts 
routinely provide before (original) and after histo- 
grams of the imagery to document the effects of apply- 
ing an image enhancement technique (Gonzalez and 
Woods, 2007; Pratt, 2007; Russ, 2011). It is instructive 
to review how a histogram of a single band of imagery, 
k, composed of i rows and j columns with a brightness 
value BV}, at each pixel location is constructed. 


Individual bands of remote sensor data are typically 
quantized (digitally recorded) with brightness values 
ranging from 2° to 2!? (if quant; = 28 then brightness 
values range from 0 to 255; 2° = values from 0 to 511; 
2!0 = values from 0 to 1,023; 2!! = values from 0 to 
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2,047; and 2!* = values from 0 to 4,095). Much of the 
remote sensor data are quantized to 8 bits, with values 
ranging from 0 to 255 (e.g., Landsat 5 Thematic Map- 
per and SPOT HRV data). Some sensor systems such 
as GeoEye-1 and MODIS obtain data with 11 bits of 
precision. The greater the quantization, the higher the 
probability that more subtle spectral reflectance (or 
emission) characteristics may be extracted from the im- 
agery (Bossler et al., 2010). 


Tabulating the frequency of occurrence of each bright- 
ness value within the image provides statistical infor- 
mation that can be displayed graphically in a 
histogram. The range of quantized values of a band of 
imagery, quant;, is provided on the abscissa (x-axis), 
while the frequency of occurrence of each of these val- 
ues is displayed on the ordinate (y-axis) (Figure 4-1a). 
For example, consider the characteristics of a Landsat 
4 Thematic Mapper image of Charleston, SC, collected 
on November 9, 1982, displayed as a color-infrared 
color composite in Figure 4-2a. Band 4 (near-infrared) 
is displayed in Figure 4-2b. A histogram of the band 4 
image data is displayed in Figure 4-2c. The peaks in the 
histogram correspond to dominant types of land cover 
in the image, including: ‘a’ open water pixels, ‘b’ coast- 
al wetland, and ‘c’ upland. Note how the Landsat The- 
matic Mapper band 4 data are compressed into only 
the lower one-third of the 0 to 255 range, suggesting 
that the data are relatively low in contrast. If the origi- 
nal Landsat Thematic Mapper band 4 brightness val- 
ues were displayed on a monitor screen or on the 
printed page they would be relatively dark and difficult 
to interpret. Therefore, in order to see the wealth of 
spectral information in the scene, the original band 4 
brightness values were contrast stretched (Figure 4- 
2b). Contrast stretching principles are discussed in 
Chapter 8. 


Histograms are useful for evaluating the quality of op- 
tical daytime multispectral data and many other types 
of remote sensor data. For example, consider the histo- 
gram of predawn thermal infrared (8.5 to 13.5 Xm) im- 
agery of a thermal plume in the Savannah River shown 
in Figure 4-3. The thermal plume entered the Savan- 
nah River via Four Mile Creek, which carried hot wa- 
ter used to cool industrial activities. The peaks in this 
histogram are associated with ‘a’ the relatively cool 
temperature of the Savannah River swamp on each 
side of the river’s natural levee, ‘b’ the slightly warmer 
temperature (12KC) of the Savannah River upstream 
and west of the plume, and ‘c’ the relatively hot ther- 
mal plume that progressed through the swamp and 
then entered the Savannah River. The north to south 
flow of the Savannah River caused the plume to hug 
the eastern bank and dissipate as it progressed away 
from the mouth of the creek. 


When an unusually large number of pixels have the 
same brightness value, the traditional histogram dis- 
play might not be the best way to communicate the in- 
formation content of the remote sensor data. When 
this occurs, it might be useful to scale the frequency of 
occurrence (y-axis) according to the relative percentage 
of pixels within the image at each brightness level 
along the x-axis. 


rN Metadata 


Metadata is data or information about data. Most 
quality digital image processing systems read, collect, 
and store metadata about a particular image or subim- 
age. It is important that the image analyst has access to 
this metadata. In the most fundamental instance, 
metadata might include: the file name, date of last 
modification, level of quantization (e.g, 8 bits), number 
of rows and columns, number of bands, univariate sta- 
tistics (minimum, maximum, mean, median, mode, 
standard deviation), georeferencing performed (if any), 
and pixel dimension (e.g., 5X 5 m). Utility programs 
within the digital image processing system routinely 
provide this information. For example, fundamental 
metadata about the Charleston, SC, Landsat 4 The- 
matic Mapper subscene and the Savannah River ther- 
mal infrared subscene are found in Figures 4-2 and 4-3, 
respectively. Neither of these images has been subject- 
ed to geometric rectification as yet. Therefore, there is 
no metadata concerning the projection, spheroid, or 
datum. This information is recorded as metadata later 
on in the process when geometric rectification takes 
place. 


An ideal remote sensing metadata system would keep 
track of every type of processing applied to each digital 
image (Lanter, 1991). This ‘image genealogy’ or ‘lin- 
eage’ information can be very valuable when the re- 
mote sensor data are subjected to intense scrutiny (e.g., 
in a public forum), used in litigation, if you want to re- 
peat a procedure, or co-register two datasets. 


The Federal Geographic Data Committee (FGDC) 
has set up rigorous image metadata standards as part 
of the National Spatial Data Infrastructure (NSDI) 
(FGDC, 2014). All federal agencies that provide re- 
mote sensor data to the public are required to use the 
established metadata standards. For example, when 
someone purchases a | X | m digital orthophotoquad 
from the U.S. Geological Survey produced from 
1:40,000-scale National Aerial Photography Program 
(NAPP) data, it comes with detailed metadata. Image 
analysts should always consult the image metadata and 
incorporate the information in the digital image pro- 
cessing that takes place. Almost all commercial remote 
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Landsat 4 Thematic Mapper Image of Charleston, SC, Collected on November 9, 1982 


Charleston 


a. Color composite of bands 4 (near-infrared), 
3 (red), and 2 (green). 


b. Band 4 (contrast stretched). 


8780 
File Info: Layer Ne Layer_4 
Last Modified: Sat Oct 27 2012 Number of Layers: 7 
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Statistics: | Median: 25 Mode: 9 Std. Dev: 15.683 
> a Skip Factor X: 2 Skip Factor Y: 2 
2 Last Modified: Sat Oct 27 2012 
= Upper Left X: 0.0 Upper Left Y: 0.0 
= Map Info: Lower Right X: 252.0 Lower Right Y: -200.0 
Pixel Size X: N/A Pixel Size Y: N/A 
Unit: other Geo. Model: None 
Projection: Unknown 
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c. Histogram of the original Landsat Thematic Mapper band 4 image. 


FIGURE 4-2 a) Landsat 4 Thematic Mapper (TM) image of Charleston, SC, obtained on November 9, 1982, displayed as a 

color-infrared color composite using bands 4 (near-infrared), 3 (red), and 2 (green). b) Display of the band 4 (near-infrared) TM 
image. c) A multimodal histogram of the brightness values of the band 4 image. Peaks in the histogram correspond to domi- 
nant types of land cover in the image, including ‘a’ open water pixels, ‘b’ coastal wetland consisting primarily of smooth cord- 


grass (Spartina alterniflora), and ‘c' upland. The inset provides basic metadata about the characteristics of the Landsat 


Thematic Mapper band 4 subscene. Original imagery courtesy of NASA. 


136 INTRODUCTORY DIGITAL IMAGE PROCESSING 


Histogram of Thermal Infrared Imagery of the Savannah River 


Savannah 
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a. Nighttime thermal infrared image of the Savannah River 
on the South Carolina and Georgia border (contrast stretched). 
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b. Histogram of the original thermal infrared data. 


FIGURE 4-3 a) Thermal infrared image of a thermal effluent plume in the Savannah River obtained on March 28, 1981, at 4:00 
a.m. b) A multimodal histogram of the brightness values in the predawn thermal-infrared image. Peaks in the histogram are 
associated with ‘a’ the relatively cool temperature of the surrounding Savannah River swamp, ‘b’ the slightly warmer temper- 
ature (12KC) of the Savannah River, and ‘c’ the relatively hot thermal plume. The ine pro is) mead taabo tb b arat er 
istics of the thermal infrared image subscene. 
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Extraction of Individual Pixel Values using the Cursor 


geographic 
area 


Cursor 


a. Nightime thermal infrared image of the Savannah River. 
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b. Individual brightness value extracted using the cursor. 


Pi 


c. Brightness values in a geographic area for an individual band. 


FIGURE 4-4 a) Thermal infrared image of the Savannah River with a cursor located at column 42 (x) and row 20 (y). b) Spec- 
tral information located at column 42 and row 20. c) Brightness values found in the first 12 columns and first 20 rows of the 


thermal infrared image displayed in a matrix format. 


sensing data providers are adopting FGDC metadata 
standards. 


The histogram and metadata information help the an- 
alyst understand the content of remotely sensed data 
and judge its quality. Sometimes, however, it is very 
useful to look at individual brightness values at specific 
locations in the imagery. 


\, Viewing Individual Pixel 
/\ Values at Specific Locations 
or within a Geographic Area 


Viewing individual pixel brightness values in a remote- 
ly sensed image is one of the most useful methods for 
assessing the quality and information content of the 
data. Virtually all digital image processing systems al- 
low the analyst to: 


* use a mouse-controlled cursor (cross-hair) to iden- 
tify a specific geographic location in the image (at a 


particular row and column or geographic x,y coor- 
dinate) and display its brightness value in n bands, 
and 


¢ display the individual brightness values of an indi- 
vidual band in a matrix (raster) format. 


Cursor Evaluation of Individual Pixel 
Brightness Values 


Most people know how to control a cursor using a 
mouse and navigate to a desired location. For example, 
a cursor has been placed in the heart of the Savannah 
River thermal plume at row 42 (x) and column 20 (y) 
in Figure 4-4a. The numeric information for this loca- 
tion in the thermal infrared image, summarized in Fig- 
ure 4-4b, includes the original brightness value (185), 
the color look-up table value (discussed in Chapter 5), 
and the number of pixels in the histogram with a value 
of 135 (1e., 7). Note the directional keys that can be 
used to navigate to nearby pixel locations if desired. 
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Two- and Three-Dimensional 
Evaluation of Pixel Brightness Values 
within a Geographic Area 


It can become quite tedious using the cursor to evalu- 
ate the individual pixel brightness values throughout 
even a small geographic area. In such circumstances, it 
is useful to identify with the cursor a geographic area 
(e.g., a rectangle) and display all the pixel values in it 
for a specific band or bands. For example, Figure 4-4c 
displays in a matrix format the brightness values found 
in the first 20 rows and 12 columns of the thermal in- 
frared image highlighted in the upper-left in Figure 4- 
4a. Brightness values >70 represent Savannah River 
pixels as well as the pixels found at the interface of the 
water and land. In Chapter 8 (Image Enhancement) we 
will perform a more rigorous examination of the ther- 
mal infrared data and select only those brightness val- 
ues >74 as being relatively pure water pixels. 


The display of individual brightness values in a matrix 
(raster) format is informative, but it does not convey 
any visual representation of the magnitude of the data 
within the area. Therefore, it is often useful early in the 
exploratory phases of a digital image processing proj- 
ect to extrude the value of the individual brightness 
values within a geographic area to create a pseudo 
three-dimensional display. For example, a wire-frame 
pseudo three-dimensional display of the individual 
brightness values over the entire thermal infrared im- 
age is shown in Figure 4-5b. While the wire-frame dis- 
play is informative, it is usually more visually effective 
to drape the actual grayscale image of the individual 
band (or bands if a color composite is being used) over 
the exaggerated three-dimensional representation of 
the brightness values found within the specific geo- 
graphic area. For example, a pseudo three-dimensional 
display of the thermal infrared image viewed from two 
different vantage points is shown in Figure 4-5c. 


The examination of histograms and the extraction of 
individual brightness values using a cursor or geo- 
graphic area analysis methods are quite useful. Howev- 
er, they provide no statistical information about the 
remote sensor data. The following sections describe 
how fundamental univariate and multivariate statistics 
are computed for the remotely sensed imagery. 


\, Univariate Descriptive Image 
Statistics 


Most digital image processing systems can perform ro- 
bust univariate and multivariate statistical analyses of 
single- and multiple-band remote sensor data. For ex- 
ample, image analysts have at their disposal statistical 


measures of central tendency and measures of disper- 
sion that can be extracted from the imagery. 


Measure of Central Tendency in 
Remote Sensor Data 


The mode (e.g., see Figure 4-1a) is the value that occurs 
most frequently in a distribution and is usually the 
highest point on the curve (histogram). It is common, 
however, to encounter more than one mode in a re- 
mote sensing dataset, such as that shown in Figure 4- 
1b. The histogram of the Landsat TM band 4 image of 
Charleston, SC (Figure 4-2c) and the predawn thermal 
infrared image of the Savannah River (Figure 4-3b) 
have multiple modes. They are referred to as being 
multimodal histograms (Figure 4-1b). They are non- 
symmetrical (skewed) distributions. 


The median is the value midway in the frequency distri- 
bution (e.g., see Figure 4-la). One-half of the area be- 
low the distribution curve is to the right of the median, 
and one-half is to the left. The mean (X) is the arithme- 
tic average and is defined as the sum of all brightness 
value observations divided by the number of observa- 
tions. It is the most commonly used measure of central 
tendency. The mean of a single band of imagery, X;, 
composed of 7 brightness values (BV;) is computed 
using the formula: 


yp, = EL—. (4.1) 


The sample mean, X,, is an unbiased estimate of the 
population mean. For symmetrical distributions the 
sample mean tends to be closer to the population mean 
than any other unbiased estimate (such as the median 
or mode). Unfortunately, the sample mean is a poor 
measure of central tendency when the set of observa- 
tions is skewed or contains an extreme value (outlier). 
As the peak (mode) becomes more extremely located 
to the right or left of the mean, the frequency distribu- 
tion is said to be skewed. A frequency distribution 
curve (histogram) is said to be skewed in the direction 
of the longer tail. Therefore, if a peak (mode) falls to 
the right of the mean, the frequency distribution is neg- 
atively skewed. If the peak falls to the left of the mean, 
the frequency distribution is positively skewed. Exam- 
ples of positively and negatively skewed distributions 
are shown in Figures 4-1d and e. 


Measures of Dispersion 


Measures of the dispersion about the mean of a distri- 
bution also provide valuable information about the im- 
age. For example, the range of a band of imagery 
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Thermal Infrared Image Draped over Three-Dimensional Representation of the Data 
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a. Thermal infrared image of the Savannah River. b. Wire-frame pseudo 3D representation of the data. 


c. Thermal infrared data draped over the 3D representation (5x exaggeration). 


FIGURE 4-5 Two- and three-dimensional views of the Savannah River thermal infrared data shown in Figure 4-3a. 
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(range;) is computed as the difference between the | Unfortunately, when the minimum or maximum values 


maximum (max;) and minimum (min,) values: are extreme or unusual observations (1.e., possibly data 
blunders), the range could be a misleading measure of 
range, = max,—min,. (4.2) dispersion. Such extreme values are common because 


the remote sensor data are often collected by detector 
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Areas Under the Normal Curve for Various 
Standard Deviations from the Mean 
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FIGJ RE 46 The area under the normal curve for a) 1, b) 2, and c) 3 standard deviations. 


systems with delicate electronics that can experience 
spikes in voltage and other unfortunate malfunctions. 
When unusual values are not encountered, the range is 
a very important statistic often used in image enhance- 
ment functions such as min—max contrast stretching 
discussed in Chapter 8. 


The variance of a sample is the average squared devia- 
tion of all possible observations from the sample mean. 
The variance of a band of imagery, var;, is computed 


using the equation: 
nN 


> (B ka ii 


var, = Se Ee (4.3) 


The numerator of the expression, y(BV;,- u,), is the 
corrected sum of squares (SS) (Davis, 2002). If the 
sample mean (X,) was actually the population mean, 
this would be an accurate measurement of the vari- 
ance. Unfortunately, there is some underestimation 
when variance is computed using Equation 4.3 because 
the sample mean XM, in Equation 4.1) was calculated in 
a manner that minimized the squared deviations about 
it. Therefore, the denominator of the variance equation 


is reduced to n — 1, producing a somewhat larger, unbi- 
ased estimate of the sample variance: 


SS 


n—-1- 


(4.4) 


vary, = 


The standard deviation is the positive square root of the 
variance. The standard deviation of the pixel bright- 
ness values in a band of imagery, s;, is computed as: 

(4.5) 


Sh — vary, 


A small standard deviation suggests that observations 
are clustered tightly around a central value. Converse- 
ly, a large standard deviation indicates that values are 
scattered widely about the mean. The total area under- 
neath a distribution curve is equal to 1.00 (or 100%). 
For normal distributions, 68% of the observations lie 
within —1 standard deviation of the mean (Figure 4- 
6a), 95.4% of all observations lie within —2 standard 
deviations (Figure 4-6b), and 99.73% within —3 stan- 
dard deviations (Figure 4-6c). 


The standard deviation is a statistic commonly used in 
digital image processing (e.g., linear contrast enhance- 
ment, spatial filtering, parallelepiped classification, 
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riance matrix 


Pixel Band 1 Band 2 Band 3 Band 4 
(green) (red) (near-infrared) (near-infrared) 

(U)) 130 Sy 180 205 

(1,2) 165 35 215 255 

(G3) 100 25 ds5 195 

(1,4) 135 50 200 220 

(FS) 145 65 205 235 


and accuracy assessment) (e.g., Schowengerdt, 2007; 
Russ, 2011; Richards, 2013; Chityala and Pudipeddi, 
2014). To interpret variance and standard deviation, 
analysts should not attach a significance to each nu- 
merical value but should compare one variance or 
standard deviation to another. The sample having the 
largest variance or standard deviation has the greater 
spread among the brightness values of the observa- 
tions, provided all the measurements were made in the 
same units. 


Measures of Distribution (Histogram) 
Asymmetry and Peak Sharpness 


Sometimes it is useful to compute additional statistical 
measures that describe in quantitative terms various 
characteristics of the distribution (histogram). Skew- 
hess is a measure of the asymmetry of a histogram and 
is computed using the formula (Pratt, 2007): 


~ (BV 4 - Hy)? 
3 aera 
ial eee eee (4.6) 


skewness, = 
n 


A perfectly symmetric histogram has a skewness value 
of zero. 


A histogram can be symmetric but have a peak that is 
very sharp or one that is subdued when compared with 
a perfectly normal distribution. A perfectly normal fre- 
quency distribution (histogram) has zero kurtosis. The 
greater the positive kurtosis value, the sharper the peak 
in the distribution when compared with a normal his- 
togram. Conversely, a negative kurtosis value suggests 
that the peak in the histogram is less sharp than that of 
a normal distribution. Kurtosis is computed using the 
formula: 


n 


BV.-—w,\" 
aoa ae (4.7) 


. 1 i 
kurtosis, = |- 
ae 


i=1 


Outliers or blunders in the remotely sensed data can 
have a serious impact on the computation of skewness 
and kurtosis. Therefore, it is desirable to remove (or re- 
pair) bad data values before computing skewness or 
kurtosis. 


( Multivariate Image Statistics 


Remote sensing research is often concerned with the 
measurement of how much radiant flux is reflected or 
emitted from an object in more than one band (e.g., in 
red and near-infrared bands). It is useful to compute 
multivariate statistical measures such as covariance 
and correlation among the several bands to determine 
how the measurements covary. Later it will be shown 
that variance—covariance and correlation matrices are 
used in remote sensing principal components analysis 
(PCA), feature selection, classification and accuracy 
assessment (e.g., Bishop, 2007; Congalton and Green, 
2008; Prost, 2013; Richards, 2013) (Chapters 8 through 
13). For this reason, we will examine how the variance— 
covariance between bands is computed and then pro- 
ceed to compute the correlation between bands. Al- 
though performed on a simple dataset consisting of 
just five hypothetical pixels (Table 4-1), this example 
provides insight into the utility of these statistics for 
digital image processing purposes. Later, these statis- 
tics are computed for a seven-band Charleston, SC, 
Thematic Mapper scene consisting of 240 H 256 pixels. 
Note that this scene is much larger than the 120 X 100 
pixel subscene in Figure 4-2a. 


The following examples are based on an analysis of the 
first five pixels [(1, 1), (1, 2), (1, 3), 1, 4) and (1, 5)Jina 
four-band (green, red, near-infrared, near-infrared) hy- 
pothetical multispectral dataset obtained over vegetat- 
ed terrain. Thus, each pixel consists of four spectral 
measurements (Table 4-1). Note the low brightness val- 
ues in band 2 caused by plant chlorophyll absorption 
of red light for photosynthesis. Increased reflectance of 
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TABLE 4-2 Univariate statistics for the hypothetical sample 
dataset. 


Band 1 2 3 4 
Mean (&,) 135.00 46.40 187.0 222.00 
Standard 23.71 16.27 31.4 23.87 


deviation (s;,) 


Variance (var) 562.50 264.80 1007.5 570.00 
Minimum (min,) 100.00 25.00 135.0 195.00 
Maximum (max;,) 165.00 65.00 215.0 255.00 
Range (BV,) 65.00 40.00 80.0 60.00 


the incident near-infrared energy by the green plant re- 
sults in higher brightness values in the two near-infra- 
red bands (3 and 4). Although it is a small hypothetical 
sample dataset, it represents well the spectral charac- 
teristics of healthy green vegetation. 


The simple univariate statistics for such data are usual- 
ly reported as shown in Table 4-2. In this example, 
band 2 exhibits the smallest variance (264.8) and stan- 
dard deviation (16.27), the lowest brightness value 
(25), the smallest range of brightness values (65 — 25 = 
40), and the lowest mean value (46.4). Conversely, 
band 3 has the largest variance (1007.5) and standard 
deviation (31.74) and the largest range of brightness 
values (215 — 135 = 80). These univariate statistics are 
of value but do not provide useful information con- 
cerning whether the spectral measurements in the four 
bands vary together or are completely independent. 


Covariance in Multiple Bands of 
Remote Sensor Data 


The different remote sensing—derived spectral measure- 
ments for each pixel often change together in a predict- 
able fashion. If there is no relationship between the 
brightness value in one band and that of another for a 
given pixel, the values are mutually independent; that 
is, an increase or decrease in one band’s brightness val- 
ue is not accompanied by a predictable change in an- 
other band’s brightness value. Because spectral 
measurements of individual pixels may not be indepen- 
dent, some measure of their mutual interaction is need- 
ed. This measure, called the covariance, is the joint 
variation of two variables about their common mean. 
To calculate covariance, we first compute the corrected 
sum of products (SP) defined by the equation (Davis, 
2002): 


TABLE 4-3 Format of a variance—covariance matrix. 


Band 1 Band 2 Band 3 Band 4 
Band 1 SS, COVy 2 COV 3 COVy 4 
Band 2 COV? 4 SS> COV? 3 COV? 4 
Band 3 COV3 4 COV3 2 SS3 COV3 4 
Band 4 cova 4 cov4 2 COV4 3 SS4 
n 
SP.) = > (BV;,—u,)(BV;;— )) (4.8) 


i=1 


In this notation, BV;, is the ith measurement of band 
k, and BV;;1s the ith measurement of band / with n pix- 
els in the study area. The means of bands k and / are X, 
and K), respectively. In our example, variable k might 
stand for band 1 and variable / could be band 2. It is 
computationally more efficient to use the following for- 
mula to arrive at the same result: 


n n 
, BV.) BV, 


a j=] i=1 
Sys) G8 = 


. (4.9) 
i=1 
The quantity is called the uncorrected sum of products. 


The relationship of SP;; to the sum of squares (SS) can 
be seen if we take k and / as being the same, that is: 


n n 
7 BV, BV ix 
oe 
> G7, x87,)-= 410 
=] 


SP. 


= SS,. 


Just as simple variance was calculated by dividing the 
corrected sums of squares (SS) by (7 — 1), covariance is 
calculated by dividing SP by (7 — 1). Therefore, the co- 
variance between brightness values in bands k and /, 
cov;), 1s equal to (Davis, 2002): 


SP_) 


n—-1 


SS, = (4.11) 


The sums of products (SP) and sums of squares (SS) 
can be computed for all possible combinations of the 
four spectral variables in Table 4-1. These data can be 
arranged in a 4 XM 4 variance—covariance matrix, as 
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TABLE 4-4 Variance-covariance matrix of the sample data. 


Band 1 Band 2 Band 3 Band 4 
Band 1 562.50 
Band 2 135.00 264.80 
Band 3 718.75 21525) 1007.50 
Band 4 537.50 64.00 663.75 570.00 


shown in Table 4-3. All elements in the matrix not on 
the diagonal have one duplicate (¢.g., cov) » = cov; so 
that cov;; = cov;;). 


The computation of variance for the diagonal elements 
of the matrix and covariance for the off-diagonal ele- 
ments of the data is shown in Table 4-4. The manual 
computation of the covariance between band | and 
band 2 is shown in Table 4-5. 


Correlation between Multiple Bands 
of Remotely Sensed Data 


To estimate the degree of interrelation between vari- 
ables in a manner not influenced by measurement 
units, Pearson’s product-moment correlation coefficient 
(r) is commonly computed (Samuels et al., 2011; Koni- 
shi, 2014). For example, the correlation between two 
bands (k and /) of remotely sensed data, r;,;, is the ratio 
of their covariance (cov,;) to the product of their stan- 
dard deviations (s;5/): 


coV 
i= — ; (4.12) 

k°l 
Because the correlation coefficient is a ratio, it is a 
unitless number. Covariance can equal but cannot ex- 
ceed the product of the standard deviation of its vari- 
ables, so correlation ranges from +1 to -l. A 
correlation coefficient of +1 indicates a positive, per- 
fect relationship between the brightness values in two 
of the bands (i.e., as one band’s pixels increase in value, 
the other band’s values also increase in a systematic 
fashion). Conversely, a correlation coefficient of —1 in- 
dicates that the two bands are perfectly inversely relat- 
ed (i.e., as brightness values in one band increase, 
corresponding pixels in the other band systematically 

decrease in value). 


A continuum of less-than-perfect relationships exists 
between correlation coefficients of -1 and +1. A corre- 
lation coefficient of zero suggests that there is no linear 
relationship between the two bands of remote sensor 
data. 
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TABLE 4-5 Computation of covariance between bands 1 
and 2 of the sample data using Equations 4.9 and 4.11. 


Band 1 (Band 1 & Band 2) Band 2 

130 7,410 Sy/ 

165 5.715 35 

100 2,500 25) 

135 6,750 50 

145 9,425 65 

675 31,860 232 
where SP, 5 = (31,860) - C7282) = By 

COv4 5 = os = 135 


If we square the correlation coefficient (r;)), we obtain 
the sample coefficient of determination (77), which ex- 
presses the proportion of the total variation in the val- 
ues of “band 7” that can be accounted for or explained 
by a linear relationship with the values of the random 
variable “band k.” Thus, a correlation coefficient (r;)) 
of 0.70 results in an 7? value of 0.49, meaning that 49% 
of the total variation of the values of “band /” in the 
sample is accounted for by a linear relationship with 
values of “band k.” 


The between-band correlations are usually stored in a 
correlation matrix, such as the one shown in Table 4-6 
that contains the between-band correlations of our 
sample data. Usually, only the correlation coefficients 
below the diagonal are displayed because the diagonal 
terms have a value of 1.0 and the terms above the diag- 
onal are duplicates. 


TABLE 4-6 Correlation matrix of the sample data. 


Band 1 Band 2 Band 3 Band 4 
Band 1 — 
Band 2 0.35 — 
Band 3 O95 0.53 — 
Band 4 0.94 0.16 0.87 _— 
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TABLE 4-7 Statistics for the Charleston, SC, Landsat Thematic Mapper scene composed of seven bands of 240 M 256 pixels 


each. 
Band 
Number 1 2 3 4 5 7 6 
(Xm) 0.45-0.52 | 0.52-0.60 | 0.63-0.69 | 0.76-0.90 | 1.55-1.75 | 2.08-2.35 | 10.4-12.5 
Univariate Statistics 
Mean 64.80 25.60 23.70 27.30 32.40 15.00 110.60 
Standard 10.05 5.84 8.30 15.76 23.85 12.45 4.21 
deviation 
Variance 100.93 34.14 68.83 248.40 568.84 154.92 17.78 
Minimum 51.00 17.00 14.00 4.00 0.00 0.00 90.00 
Maximum 242.00 115.00 131.00 105.00 193.00 128.00 130.00 
Variance—Covariance Matrix 
1 100.93 
2 56.60 34.14 
3 79.43 46.71 68.83 
4 61.49 40.68 69.59 248.40 
5 134.27 85.22 141.04 330.71 568.84 
7 90M3' 55.14 86.91 148.50 280.97 154.92 
6 23.72 14.33 2292 43.62 78.91 42.65 WETS 
Correlation Matrix 
1 1.00 
2 0.96 1.00 
&) 0.95 0.96 1.00 
4 0.39 0.44 0.53 1.00 
5 0.56 0.61 0.71 0.88 1.00 
/ OFZ 0.76 0.84 0.76 0.95 1.00 
6 0.56 0.58 0.66 0.66 0.78 0.81 1.00 


In this hypothetical example, brightness values of band 
1 are highly correlated with those of bands 3 and 4, 
that is, r > 0.94. A high correlation suggests substantial 
redundancy in the information content among these 
bands. Perhaps one or more of these bands could be 


deleted from the analysis to reduce subsequent compu- 
tation. Conversely, the relatively lower correlation be- 
tween band 2 and all other bands suggests that this 
band provides some type of unique information not 
found in the other bands. More sophisticated methods 
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of selecting the most useful bands for analysis are de- 
scribed in later sections. 


The results of performing a typical statistical analysis 
program on the Charleston, SC, Landsat TM data are 
summarized in Table 4-7. Band | exhibits the greatest 
range of brightness values (from 51 to 242) due to Ray- 
leigh and Mie atmospheric scattering of blue wave- 
length energy. The near- and middle-infrared bands (4, 
5, and 7) all have minimums near or at zero. These val- 
ues are low because much of the Charleston scene is 
composed of open water, which absorbs much of the 
incident near- and middle-infrared radiant flux, there- 
by causing low reflectance in these bands. Bands 1, 2, 
and 3 are all highly correlated with one another (r > 
0.95), indicating that there is substantial redundant 
spectral information in these channels. Although not 
to the same degree, there is also considerable redun- 
dancy among the reflective and middle-infrared bands 
(4, 5, and 7), as they exhibit correlations ranging from 
0.66 to 0.95. Not surprisingly, the lowest correlations 
occur when a visible band is compared with an infrared 
band, especially bands | and 4 (r = 0.39). In fact, band 
4 is the least redundant infrared band when compared 
with all three visible bands (1, 2, and 3). For this rea- 
son, TM band 4 (0.76 to 0.90 Xm) is used as an exam- 
ple throughout much of the text. As expected, the 
thermal-infrared band 6 data (10.4 to 12.5 Mm) are 
highly correlated with the middle-infrared bands (5 
and 7). 


( Feature Space Plots 


The univariate and multivariate statistics discussed 
provide accurate, fundamental information about the 
individual band statistics including how the bands co- 
vary and correlate. Sometimes, however, it is useful to 
examine statistical relationships graphically. 


Individual bands of remotely sensed data are often re- 
ferred to as features in the pattern recognition litera- 
ture. To truly appreciate how two bands (features) in a 
remote sensing dataset covary and if they are correlat- 
ed or not, it is often useful to produce a two-band fea- 
ture space plot. 


A two-dimensional feature space plot extracts the 
brightness value for every pixel in the scene in two 
bands and plots the frequency of occurrence in a 255 
by 255 feature space (assuming 8-bit data). The greater 
the frequency of occurrence of unique pairs of values, 
the brighter the feature space pixel. For example, we 
know from Table 4-7 that Landsat Thematic Mapper 
bands 3 and 4 of the November 9, 1982, Charleston, 
SC, scene are not highly correlated (r = 0.53). A two- 
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dimensional feature space plot of the two bands is 
shown in Figure 4-7. The original histograms for bands 
3 and 4 are also provided. Because the two bands are 
not highly correlated, the cloud of points looks some- 
what like a tilted cap in two-dimensional feature space. 
The bright areas in the plot represent pixel pairs that 
have a high frequency of occurrence in the images. It is 
usually exciting to see feature space plots such as this 
because it suggests that there is great information of 
value in these two bands. It reveals visually that the 
two bands do not contain much redundant informa- 
tion. If the two bands were highly correlated, there 
would be significant redundant information between 
the two bands and the cloud of points would appear as 
a relatively narrow ellipse generally trending diagonally 
somewhere in the feature space between 0,0 and the 
255,255 coordinates (not shown). 


The two-dimensional feature space plot also drives 
home the point that in this particular dataset, the 
Landsat Thematic Mapper bands 3 and 4 brightness 
values are seriously constrained to the lower half of the 
possible 256 values (i.e., band 3 has a maximum value 
of 131; band 4 has a maximum value of only 4). This 
suggests that the data in both bands will probably need 
to be carefully contrast stretched if humans are going 
to be called upon to visually extract useful information 
from these individual bands. More will be said about 
the use of feature space plots in the discussion on fea- 
ture space partitioning and image classification in 
Chapter 9. 


\, Geostatistical Analysis, 
yN Autocorrelation and Kriging 
Interpolation 


A random variable distributed in space (e.g., spectral 
reflectance) is said to be regionalized. We can use geo- 
statistical analysis to extract the spatial properties of 
regionalized variables (Curran, 1988; Woodcock et al., 
1988ab; Burnicki, 2011). Once quantified, the regional- 
ized variable properties can be used in many remote 
sensing applications such as image classification (e.g., 
Maillard, 2003) and the allocation of spatially unbi- 
ased sampling sites during classification map accuracy 
assessment (e.g., Zhu and Stein, 2006; Van der Meer, 
2012). An important application of geostatistics is the 
prediction of values at unsampled locations. For exam- 
ple, one could obtain LiDAR-derived elevation values 
throughout a study area. Although they are very accu- 
rate, there may be small or large areas where LiDAR- 
derived elevation data are missing (i.e., data voids). 
Geostatistical interpolation techniques could be used 
to evaluate the spatial relationships associated with the 
existing LiDAR-derived elevation data to create a new, 
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255 


Two-Dimensional Feature Space Plot of 
Landsat Thematic Mapper Band 3 and 4 Data of Charleston, SC 


Thematic Mapper Band 4 


8697 


7577 


Thematic Mapper Band 3 


7 A two-dimensional feature space plot of Landsat Thematic Mapper bands 3 and 4 data of Charleston, SC, ob- 


tained on November 9, 1982. The original histograms of TM bands 3 and 4 are also provided. The minimum and maximum 
values within each band are bounded by a white box.The greater the frequency of occurrence of a particular pair of values in 
band 3 and band 4 in the original imagery, the brighter the pixel in the 255 X 255 feature space. For example, it appears that 
there are many pixels in the scene that have a brightness value of approximately 17 in band 3 and a brightness value of 7 in 
band 4. Conversely, there is at least one pixel in the scene with a brightness value of 131 in band 3 and 105 in band 4. Be- 
cause there is probably only one such pair, it shows up as a dark pixel in the feature space. 


improved systematic grid of elevation values. This ap- 
plication will be demonstrated. 


Geostatistical analysis is a special branch of spatial sta- 
tistics that takes into account not only the distance be- 
tween control point observations but also their spatial 
autocorrelation (Kalkhan, 2011; Jensen and Jensen, 


2013). Originally, geostatistics was synonymous with 
kriging—a statistical version of interpolation. Kriging, 
named after the work of Danie Krige (1951), is a ge- 
neric name for a family of least-squares linear regres- 
sion algorithms used to estimate the value of a 
continuous attribute (e.g., elevation) at any unsampled 
location using only the attribute data available over the 
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FIGURE 4-8 Phenomena that are geographically closer to 
one another are generally more highly correlated than phe- 
nomena that are farther apart. Geostatistical analysis incor- 
porates spatial autocorrelation information in the Kriging- 
interpolation process. 


study area (Lo and Yeung, 2006; Bachi, 2010). Howev- 
er, geostatistical analysis now includes not only kriging 
but also the traditional deterministic spatial interpola- 
tion methods. One of the essential features of geostatis- 
tics is that the phenomenon being studied (e.g., 
elevation, reflectance, temperature, precipitation, a 
land cover class) must be capable of existing continu- 
ously throughout the landscape. 


Most people are aware that things that are close to one 
another in geographic space are more alike than those 
that are farther away. Therefore, as distance increases, 
the spatial autocorrelation decreases as shown in Fig- 
ure 4-8. Kriging makes use of this spatial autocorrela- 
tion information. Kriging is similar to “distance 
weighted interpolation” in that it weights the surround- 
ing nearby values to derive a prediction for each new 
location in the output raster dataset. However, the 
weights are based not only on the distance between the 
measured control points and the point to be predicted 
(used in Inverse-Distance-Weighting), but also on the 
overall spatial arrangement among the measured points 
(i.e., their autocorrelation). This is the most significant 
difference between deterministic (traditional) and geo- 
statistical analysis. Traditional statistical analysis as- 
sumes the samples derived for a particular attribute are 
independent and not correlated in any way. Conversely, 
geostatistical analysis allows users to compute distanc- 
es between observations and to model autocorrelation 
as a function of distance and direction. This informa- 
tion is then used to refine the kriging interpolation pro- 
cess, making predictions at new locations more 
accurate than those derived using traditional methods 
such as Inverse-Distance-Weighting (Lo and Yeung, 
2006; Kalkhan, 2011; Esri, 2014). 
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Empirical Semivariogram 


Semivariance, Yp 
g. 
Q 


Lag Distance, (/) 
(e.g., in pixels or meters) 


FIGURE 49 The z-values of points (e.g., pixels in an image 
or locations on the ground if collecting in situ data) separated 
by various lag distances (h) may be compared and their semi- 
variance (Y ,) computed (based on Isaaks and Srivastava, 
1989; Lo and Yeung, 2006; Jensen and Jensen, 2013). The 
semivariance (¥ ;,) at each lag distance may be displayed as a 
s emvariogram with the range, sill, and nugget variance char- 
ateitis d id in Tabe 4-9. 


The Kriging process generally involves two operations: 


* quantifying the spatial structure of the surrounding 
data points using variography, and 


* predicting a Value;,jg at each new location in the 
output dataset. 


Variography is the process whereby a spatially depen- 
dent model is fit to the data and the spatial structure is 
quantified. To make a prediction for an unknown value 
at a specific location, Kriging uses the fitted model 
from variography, the spatial data configuration, and 
the values of the measured sample points around the 
prediction location. 


One of the most important measurements used to un- 
derstand the spatial structure of regionalized variables 
is the empirical semivariogram (Figure 4-9), which can 
be used to relate the semivariance to the amount of 
spatial separation (and autocorrelation) between sam- 
ples. The semivariance provides an unbiased descrip- 
tion of the scale and pattern of spatial variability 
throughout a region. For example, if elevation values 
of a relatively flat area are examined, there may be little 
spatial variability (variance), which will result in a 
semivariogram with predictable characteristics. Con- 
versely, the elevation found in rugged terrain may ex- 
hibit significant spatial variability resulting in an 
entirely different semivariogram. 
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Calculating Average Semivariance 

Consider the hypothetical situation where we want to 
determine the elevation of an unknown point by evalu- 
ating the spatial characteristics of six nearby points. 
The six points, Z, through Z¢, are arranged in a line in 
a Cartesian coordinate system in Table 4-8a. The rela- 
tionship between a pair of pixels / intervals apart (h is 
referred to as the lag distance; Table 4-9) can be given 
by the average variance of the differences between all 
such pairs. The unbiased estimate of the average semi- 
variance of the population (y,) is expressed through 
the relationship (Isaaks and Srivastava, 1989; Slocum 
et al., 2008): 


t= bn (4.13) 


where Z; is a control point value; / is the multiple of 
the distance between control points, and n is the num- 
ber of points. The total number of possible pairs m 
along the transect is computed by subtracting the lag 
distance / from the total number of pixels present in 
the dataset n, that is, m = n — A (Brivio and Zilioli, 
2001; Lo and Yeung, 2006). In this example, we are 
only computing the semivariance in the x-direction. In 
practice, semivariance is computed for pairs of obser- 
vations in all directions, N, NE, E, SE, S, SW, W, and 
NW (Maillard, 2003). In this manner, directional semi- 
variograms are derived and directional influences can 
be examined. 


The computation of the semivariance for the six eleva- 
tion control points is summarized in Table 4-8b. The 
semivariance (y;) of the six points when / = 1 is 17.9 
based on the following calculations: 


6-1 4 
= (Z;—Z; 4p) 
i=1 


26-1) where 


Yn 


2 2 
Yaz (Zy-Z) + (Zy—Zy) + 


2 2 2 
(Z,=Z,) +(Zy-Zs) #(Z5-Z,y /10 


y, = (10 - 15)” + (15 — 20)" + 
2 2 2 
(20 — 30)” + (30-35) + (35 —33)°/10 


ip > Te 


The average semivariance is a good measure of the 
amount of dissimilarity between spatially separate con- 
trol points. Generally, the larger the average semivari- 
ance (y ;,), the less similar are the point observations. In 
Table 4-8b we see that at a lag distance of 1, semivari- 
ance = 17.9. At a lag distance of 2, semivariance = 
69.88. At a lag distance of 3, semivariance =161.5, etc. 
In this hypothetical dataset, as the lag distance increas- 
es, there is greater dissimilarity among the six ground 
control points being examined. The greater the lag dis- 
tance, the less correlated the observations. 


Empirical Semivariogram 

The semivariogram is a plot of the average semivari- 
ance value on the y-axis with the various lags (/) inves- 
tigated on the x-axis, as shown in Figure 4-9 and Table 
4-8c. Important characteristics of the semivariogram 
include: 


¢ lag distance (/) on the x-axis, 
¢ sill (s), 

* range (a), 

* nugget variance (C,), and 


* spatially dependent structure displayed graphically 
(e.g., Figure 4-9; Table 4-8c). 


When spatial correlation exists, pairs of points that are 
close together (on the far left of the x-axis) should have 
less difference (be low on the y-axis). As points become 
farther away from each other (moving right on the x- 
axis), in general, the difference squared should be 
greater (moving up on the y-axis). The semivariogram 
model often flattens out at a certain lag distance from 
the origin. The distance where the model first flattens 
out is the range. The range is the distance over which 
the samples are spatially correlated. The si// is the value 
on the y-axis where the range is located. It is the point 
of maximum variance and is the sum of the structural 
spatial variance and the nugget effect. The partial sill is 
the sill minus the nugget. 


The semivariogram value on the y-axis should theoreti- 
cally equal zero when lag (4) equals zero. However, at 
an infinitesimally small separation distance, the semi- 
variogram often exhibits a nugget effect, which is >0. 
The nugget effect is attributed to measurement errors 
or spatial sources of variation at distances smaller than 
the sampling interval (or both). 


A semivariogram provides information on the spatial 
autocorrelation of control point datasets. For this rea- 
son and to ensure that Kriging predictions have posi- 
tive Kriging variances, a model is fit to the 
semivariogram (i.e., a continuous function or curve). 
This model quantifies the spatial autocorrelation in the 
data (Johnston et al., 2001). 
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TABLE 4-8 Semivariance computation (adapted from Jensen and Jensen, 2013). 


a. Six equally-spaced hypothetical elevation control points (Z; through Z,). 


A 


lag, = _@$jNHNA90._ I pair 


lag =4.- 
| 


lag =3 1 


_... —w 
 —— rr 


lag =2 4 


lag =| ——7—_1—_ 1 5S pairs 


Z, 2, 23 
o° © © 
10 15 20 


2 pairs 


3 pairs 
———a | 


4 pairs 
Zy 25 Z6 
O° ° fe) 
30035 33 
| | | >x 


10 20 30 


40 50 60 


b. Semivariance computations. 


h 
1 2 3 4 5 
Z-Zaw 25 100 400 625 529 
a a 25 225 400 324 
Cre 100 225 169 
(24-24, i 25 9 
(Z5~Z5 <1). 4 
“= 2 
24 fw 179 559 969 949 529 
2(n—h) 10 8 6 4 2 
Th 17.9 69.88 161.5 237.25 264.5 


h=\1 h=2 


h=3 


Distance between control points 
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It iging Spatial Interpolation of LiDAR Masspoints, An alytical Hill-shading, 
Hypsometric Tinting, and @ ntour Extraction 


Semivariogram of the LiDAR Masspoints 


Range = 29.47 


wee Spee 


Partial sill = 0.824} 


Nugget =0 Lag = 42 8 Lags =11 


0.75 15° | 2:25 3 3.75 45 5.25 
Distance, / x 10°! 


b. Semivariogram of the LIDAR masspoints. 


c. Analytically hill-shaded surface created using Kriging. 


Elevation 
in meters 


[| 86.50 - 86.74 
[] 86.75 - 86.99 
[__] 87.00 - 87.24 
[) 87.25 - 87.49 
HN 87.50 - 87.74 
WM 87.75 - 87.99 
GN 88.00 - 88.24 
88.25 - 88.49 
HN 88.50 - 88.74 
MM 88.75 - 88.99 
HE 89.00 - 29.24 
HM 89.25 - 89.49 
HEM 89.50 - 89.74 
HM 89.75 - 89.99 
HE (90.00 - 90.24 
90.25 - 90.49 
[ 90.50 - 90.74 
[| 90.75 - 90.99 
[| 91.00 - 91.24 


VAN 


eee 


d. Kriging-derived elevation surface hypsometrically tinted using 19 class intervals and overlaid with 25 cm contours. 


FIGURE 4-10 a) 37,150 LIDAR masspoints (green dots) overlaid on a 1% 1 m color-infrared orthophotograph. b) A semivario- 
gram of the 37,150 LIDAR masspoints processed using spherical Kriging. This semivariogram contains autocorrelation infor- 
mation derived from all orientations (e.g., N, E, S, and W). c) An analytically hill-shaded surface created using a Kriging model. 
The cell size is 0.25 M 0.25 m. d) The Kriged surface color-coded using 19 class intervals and overlaid with 25 cm contours de- 
rived from the elevation surface. 
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TABLE 4-9 Terminology and symbols in a typical empirical semivariogram (based on Curran, 1988; Johnston et al., 2001; 
Lo and Yeung, 2006; Jensen and Jensen, 2013). 


Term Symbol Definition 
h The linear (horizontal) distance that separates any two locations (i.e., a sampling pair). A lag has 

Lag length (distance) and direction (orientation). 

The maximum level of the modeled semivariogram. The value that the variogram tends to when 
Sill s lag distances become very large. At large lag distances, variables become uncorrelated, so the sill 

of the semivariogram is equal to the variance of the random variable. 

Point on the h-axis where the modeled semivariogram nears a maximum. The distance beyond 
Range a which there is little or no autocorrelation among variables. Places closer than the range are auto- 

correlated, places farther apart are not. 

The location where the modeled semivariogram intercepts the ¥ (h) axis. Represents the indepen- 
Nugget Cy dent error, measurement error, or microscale variation at spatial scales that are too fine to detect. 
variance The nugget effect is a discontinuity at the origin of the semivariogram model. 
Partial sill Cc The sill minus the nugget variance describes the spatially dependent structural variance. 


The line fitted to the data points in Figure 4-9 and in 
Table 4-8c is called a semivariogram model. The model 
is a line through the average semivariance versus aver- 
age distance between point pairs. The user selects the 
functional form that best represents the distribution 
(e.g., spherical, circular, etc.). The coefficients of the 
function are empirically derived from the data. 


It is possible to work with relatively few or many obser- 
vations to predict the value at unknown locations using 
Kriging. For example, consider a relatively large Li- 
DAR dataset consisting of 37,150 masspoints located 
on the Savannah River Site shown in Figure 4-10a. The 
semivariogram shown in Figure 4-10b contains de- 
tailed information about the range, partial sill, and 
nugget associated with the entire dataset. A spherical 
geostatistical Kriging algorithm was used to create the 
raster continuous surface (Figure 4-10c) from the 
37,150 LiDAR masspoints shown in Figure 4-10a. The 
raster surface has been analytically hill-shaded. A col- 
or-coded version of the Kriged elevation surface with 
19 class intervals is shown in Figure 4-10d overlaid 
with derived 25 cm contours. 
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5 DISPLAY ALTERNATIVES AND 
SCIENTIFIC VISUALIZATION 


Scientists interested in displaying and analyzing re- 
motely sensed data actively participate in scientific vi- 
sualization, defined as “visually exploring data and 
information in such a way as to gain understanding 
and insight into the data.” The difference between sci- 
entific visualization and presentation graphics is that 
the latter are primarily concerned with the communica- 
tion of information and results that are already under- 
stood. During scientific visualization we are seeking to 
understand the data and gain new insight (Earnshaw 
and Wiseman, 1992; Slocum et al., 2008; Myler, 2013). 


Scientific visualization of remotely sensed data is ma- 
turing. Its origin can be traced to the simple plotting of 
points and lines and contour mapping (Figure 5-1). We 
now have the ability to conceptualize and visualize re- 
motely sensed images in two-dimensional space in true 
color as shown (two-dimensional to two-dimensional). 
It is also possible to drape remotely sensed data over a 
digital elevation model (DEM) and display the syn- 
thetic three-dimensional model on a two-dimensional 
map or computer screen (i.e., three-dimensional to 
two-dimensional). If we transformed this same three- 
dimensional model into a physical model that we could 
touch, it would occupy the three-dimensional to three- 
dimensional portion of scientific visualization mapping 
space. 


es Overview 


This chapter identifies the challenges and limitations 
associated with displaying remotely sensed data and 
makes suggestions about how to display and visualize 


the data using black-and-white and color output devic- 
es. The characteristics of 8- and 24-bit color look-up 
tables are presented. Methods of selecting the most ap- 
propriate bands to display in a color composite display 
are presented (e.g., optimum index factor, Sheffield In- 
dex). Methods of merging (fusing) different types of re- 
mote sensor data are discussed (eg., component 
substitution) along with several color coordinate sys- 
tems (e.g., RGB, intensity—hue—saturation [THS], and 
chromaticity). Methods of extracting distance and 
polygon perimeter, area, and shape information from 
remotely sensed images are discussed. 


L 3 Image Display Considerations 


Humans are very adept at visually interpreting contin- 
uous-tone images every day as they read magazines 
and newspapers or watch television (Ready, 2013). Our 
goal is to capitalize on this talent by providing remote- 
ly sensed data in a format that can be easily visualized 
and interpreted to gain new insight about the Earth 
(Jensen and Jensen, 2013). The first problem is that the 
remotely sensed data collected by government agencies 
(e.g., NASA Landsat 8 data) or private industry (e.g., 
GeoEye, Inc., DigitalGlobe, Inc., and SPOT Image, 
Inc.) are in a digital format. How do we convert the 
brightness values (BVs) stored on any type of storage 
device into an image that begins to approximate the 
continuous-tone photographs so familiar to humans? 
The answer is the creation of a brightness map, also 
commonly referred to as a grayscale or color image. 


A brightness map is a computer graphic display of the 
brightness values, BV;;;, found in digital remotely 
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FIGURE 5-1 Scientific visualization mapping space. The x-axis is the conceptual domain or how we conceive the information 
in our minds. The y-axis is the actual number of dimensions used to visually represent our conceptual ideas (based on 


Earnshaw and Wiseman, 1992). 


sensed data (refer to Figure 2-1). Ideally, there is a one- 
to-one relationship between input brightness values 
and the resultant intensities of the output brightness 
values on the display as shown in Figure 5-2a. For ex- 
ample, an input BV of 0 would result in a very dark 
(black) intensity on the output brightness map, while a 
BV of 255 would produce a bright (white) intensity. All 
brightness values between 0 and 255 would be dis- 
played as a continuum of grays from black to white. In 
such a system, an input brightness value of 127 would 
be displayed exactly as 127 (mid-gray) in the output 
image, as shown in Figure 5-2a (assuming contrast 
stretching does not take place). Unfortunately, it is not 
always easy to maintain this ideal relationship. In the 
past, it was common for analysts to have access to de- 
vices that displayed only a relatively small range of 
brightness values (e.g., <50) (Figure 5-2b). In this ex- 
ample, several input brightness values around BV 127 
might be assigned the same output brightness value of 
25. When this occurs, the original remotely sensed data 
are generalized when displayed, and valuable informa- 
tion may never be seen by the image analyst. Therefore, 
it is important that whenever possible the one-to-one 
relationship between input and output brightness val- 
ues be maintained. 


This chapter describes the creation of remote sensing 
brightness maps using two fundamentally different 
output devices: hard-copy displays and temporary vid- 
eo displays. Hard-copy displays are based on the use of 
line printers, line plotters, ink-jet printers, laser print- 
ers, or film writers to produce tangible hard copies of 
the imagery for visual examination. Temporary video 
displays are based on the use of black-and-white or col- 
or video technology, which displays a temporary image 
for visual examination. The temporary image can be 
discarded or subsequently routed to a hard-copy de- 
vice if desired. 


Hard-copy image displays may be produced using line 
printer/plotters, laser printers, or ink-jet printers. 


Black-and-White Hard-Copy 
Image Display 


Line Printer/Plotter Brightness Maps 


At one time, the 6- or 8-line per inch alphanumeric line 
printer was the device most often used to produce 
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FIGURE 5-2 a) An ideal one-to-one relationship between 8-bit input remote sensing brightness values and the output bright- 
ness map. Most modern computer graphic and digital image processing workstation environments maintain this relationship. 
b) A poor situation where an analyst's output device is not capable of retaining the one-to-one relationship and must general- 
ize the original 8-bit data down to a more manageable number of brightness map classes. In this case, the output device has 
only 50 classes. If the 8-bit data were uniformly distributed on the x-axis, >5x reduction of information would take place using 
this hypothetical output device. These two examples assume that the remotely sensed data are not contrast stretched. 
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FIGURE 5-3 The brightness value of a picture element (pixel) is read from mass storage by the central processing unit (CPU). 
The digital value of the stored pixel is in its proper i,j location in the image processor's random-access memory (RAM), often 
referred to as a video frame buffer. The brightness value is then passed through a black-and-white or color look-up table 
where modifications can be made. The output from the digital color look-up table is passed to a digital-to-analog converter 
(DAC). The output from the DAC determines the intensity of the signal for the three guns (red, green, and blue) in the back of 
the monitor that stimulate the phosphors on a computer cathode-ray tube (CRT) or at a specific x, y location or the transistors 
in a liquid crystal display (LCD). A DAC is not required if a digital monitor is used. 


scaled hard-copy images. The input data were density 
sliced into a series of discrete class intervals in which 
each interval corresponded to a specific brightness val- 
ue range. To produce density-sliced maps on the line 
printer, it was necessary to select 1) an appropriate 
number of class intervals, 2) the size or dimension of 
each class interval, and 3) the alphanumeric symbol- 
ization to be assigned to each class interval. Some- 
times, more sophisticated line plotters were available. 
Line plotters were programmed to give the impression 
of continuous tone using crossed-line shading pro- 
duced by intertwining two perpendicular sets of equal- 
ly spaced parallel lines. Alphanumeric printers and line 
plotters are still used to produce hard-copy output if 
laser or ink-jet printers are not available. 


Laser or Ink-Jet Printer Brightness 
Maps 


Most image analysts now use relatively inexpensive la- 
ser or ink-jet printers to output what appears to be 


continuous-tone black-and-white or color imagery. 
This is accomplished by the system’s software, which 
develops a functional relationship between a pixel’s in- 
put brightness value and the amount of laser toner or 
ink-jet ink that is applied (output) at the appropriate 
location on the printed page. These relatively inexpen- 
sive printers can normally apply the toner or ink-jet 
ink at 100 to 1,200 dots per inch (dpi). 


Temporary Video Image 
Display 


The most effective display of remote sensor data is 
based on the use of temporary video displays that have 
improved black-and-white and color brightness map 
display capabilities. 
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TABLE 5-1 Compression scheme characteristics for selected raster image file formats (based on Myler, 2013). 


(Graphic interchange format) 


Format Compression Scheme 
Raw None—pixel values are stored directly. For example, ERDAS and ENVI software stores the 
remote sensor data in its raw, original format with no compression. 
GIF 


Lossless for grayscale images. Can be lossy for color images. 


JPEG 
(Joint photographic experts group) 


Inherently lossy, JPEG removes redundant data based on a model of human color percep- 
tion. JPEG can also be used to store a lossless image, but the file size will be greater than 
that of the raw image because additional information is added to the file. 


(Tagged interchange file format) 


TIFF Compression scheme selectable but most schemes used are lossless. TIFF file is usually larger 
than the raw file because additional information is added to the file. 


Black-and-White and Color 
Brightness Maps 


A video image display of remotely sensed data can be 
easily modified or discarded. The central processing 
unit (CPU) reads the digital remote sensor data from a 
mass storage device (e.g., hard disk or optical disk) and 
transfers these data to the image processor’s random 
access memory (RAM) frame buffer (Figure 5-3). The 
image processor frame buffer is a collection of display 
memory composed of i lines by 7 columns and 5 bits 
that can be accessed sequentially, line by line. Each line 
of digital values stored in the image processor display 
memory is continuously scanned by a read mechanism. 
The content of the image processor memory is read ev- 
ery 1/60 of a second, referred to as the refresh rate of 
the system. The brightness values encountered during 
this scanning process are passed to the color look-up 
table (LUT). An analyst can change the contents of the 
color look-up table to modify how an individual pixel 
eventually appears on the computer screen. The con- 
tents of the look-up table are then passed to a digital- 
to-analog converter (DAC) that prepares an analog 
video signal suitable for display on the cathode-ray 
tube (CRT) or liquid crystal display (LCD) (Myler, 
2013). Thus, the analyst viewing a computer screen is 
actually looking at the video expression of the digital 
values stored in the image processor’s memory. The re- 
fresh rate (1/60 second) is so fast that the analyst does 
not see any significant amount of flicker on the screen. 
If a digital display is used (as opposed to an analog dis- 
play), there is no need for a digital-to-analog converter 
(DAC) because the digital data are passed directly to 
the digital display. 


Image Data Format and Compression Scheme 

The most common bitmapped graphics formats are 
raw, GIF (Graphic interchange format), JPEG (Joint 
photographic experts group), and TIFF (Tagged inter- 
change file format). Their characteristics are summa- 
rized in Table 5-1 (Myler, 2013). When remote sensor 
data are stored directly with no compression it is called 


a raw image file and is considered to be Joss/less. Con- 
versely, some image formats use a compression scheme 
to reduce the amount of data that must be stored. 
Compression schemes take advantage of the repetition 
or redundancy in neighboring pixel values. Various 
types of image data compression are presented in Table 
5-1. They can be lossless or lossy. Lossy image data 
compression schemes almost always provide good data 
reduction but at the risk of losing valuable spectral in- 
formation in the image (SPIE, 2013). Therefore, digital 
image processing analysts are strongly encouraged to 
use only Jossless image data compression schemes 
when storing their remote sensor data (e.g., in raw [ER- 
DAS, ENVI] or uncompressed TIFF format) so they 
don’t lose valuable spectral information. 


But how does the entire range of brightness values in a 
lossy or lossless digital image dataset get displayed 
properly? This is a function of 1) the number of bits 
per pixel associated with the bitmapped graphic under 
consideration, 2) the color coordinate system being 
used, and 3) the video look-up tables associated with 
the image processor. 


Bitmapped Graphics 


The digital image processing industry refers to all ras- 
ter images that have a pixel brightness value at each 
row and column in a matrix as being bitmapped images. 
The tone or color of the pixel in the image is a function 
of the value of the bits or bytes associated with the pix- 
el and the manipulation that takes place in a color 
look-up table. For example, the simplest bitmapped 
image is a binary image consisting of just ones (1) and 
zeros (0). It is useful to describe how a simple binary 
bitmapped image is encoded and displayed, before dis- 
cussing how grayscale and color images are encoded 
and displayed. 


The simple scene in Figure 5-4 consists of a house, a 
very large tree, and two road segments. The geographic 
area has been digitized into a simple raster of 1s and Os 
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Characteristics of a Binary Bitmapped Image 
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5. The bytes in the video memory that make up the black-and-white image consist of some bits set to 1 
and the rest to 0. Pixels with a | are turned on and pixels with a 0 are turned off. The scene consists of 
121 pixels, which in a binary black-and-white image can be stored as 16 bytes. The last 7 bits are 
discarded because they fall outside the matrix of pixels specified in the header. 


FIGURE 5-4 Characteristics of a binary (0 and 1), black-and-white bitmapped image. 


and stored in a bitmapped format as a unique graphics 
file on a hard disk or other mass storage device. How 
do we read the contents of the bitmapped graphic file 
and construct a digital image from the bits and bytes 
stored in it? First, all raster graphic files contain a 
header. The first record in the header file is the signa- 
ture, which lets the digital image processing program 
(application) know what type of file is present. The 
next record in the header identifies the number of rows 
(height) and columns (width) in the bitmapped dataset. 


The final records in the header contain a summary of 
the number of colors in the palette (e.g., 2) and the na- 
ture of the colors in the original palette (e.g., black and 
white). In this example, the bitmapped image contains 
11 rows and 11 columns of data with a black-and-white 
color palette. 


Based on the information in the header, the digital im- 
age processing program can extract the values of the 
121 individual picture elements in the image. Note that 
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FIGURE 5-5 Display of the Savannah River thermal infrared data at various bitmap resolutions. 


the information content of the entire 121-element ma- 
trix can be stored in just 16 bytes (Figure 5-4). Each 8- 
bit byte contains 8 individual values for 8 pixels. The 
last 7 bits of byte 16 are not used since the header re- 
cord specified that there would only be 121 useful val- 
ues, not the 128 possible values (8 bits x 16 bytes = 
128) in a 16-byte dataset. With all the information in 
hand, the program displays a binary black-and-white 
image of the scene with zeros in black and ones in 
white. 


Few remotely sensed images of significant value con- 
tain only binary (1-bit) information. Instead, most re- 
mote sensor data are quantized to 8 bits per pixel with 


the values ranging from 0 to 255 in each band. For ex- 
ample, consider the various bitmapped displays of the 
thermal plume in the Savannah River shown in Figure 
5-5. As expected, the original 8-bit display contains all 
of the information in the original dataset with a mini- 
mum value of 38 and a maximum value of 188. The 
most dramatic drop-off in visual information content 
seems to occur when the data are displayed using 4 bits 
or fewer. A 4-bit display only records values that have 
been transformed to be within the range 0 to 15. The 
binary 1-bit display contains very little useful informa- 
tion. In effect, every pixel in the 1-bit display with a 
brightness value <128 has a value of 0 and every pixel 
>128 has a value of 1. It is clear from this illustration 
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FIGURE 5-6 a) The RGB color coordinate system is based on additive color theory. If we are analyzing three 8-bit images 
(RGB) at one time, there is the possibility of 224 = 16,777,216 unique values. Each of these unique values lies somewhere with- 
in the three-dimensional RGB color coordinate system. Black-and-white grayscale images are located along the diagonal from 
0,0,0 to 255,255,255. b) Additive color theory — equal proportions of blue, green, and red light superimposed on top of one 
another creates white light, i.e., white light is composed of blue, green, and red light. The complementary colors yellow, ma- 
genta, and cyan are created by selectively adding together red and green, blue and red, and blue and green light, respective- 
ly. c) Subtractive color theory — equal proportions of blue, green, and red pigments yield a black surface. A yellow filter 
effectively absorbs all blue light, a magenta filter absorbs all green light, and a cyan filter absorbs all red light. 


that whenever possible we should use at least 8-bit bit- 
mapped displays when analyzing 8-bit remotely sensed 
data. 


The remainder of the discussion is based on the use of 
8-bit imagery. However, numerous remote sensing sys- 
tems now have 9-, 10-, 11-, and even 12-bit radiometric 
resolution per band. Therefore, it is important not to 
take this parameter for granted. 


RGB Color Coordinate System 


Digital remote sensor data are usually displayed using 
a Red-Green-Blue (RGB) color coordinate system (Fig- 
ure 5-6a), which is based on additive color theory and 
the three primary colors of red, green, and blue (Figure 
5-6b). Additive color theory is based on what happens 
when light is mixed, rather than when pigments are 
mixed using subtractive color theory. For example, in 
additive color theory a pixel having RGB values of 
255, 255, 255 produces a bright white pixel (Figure 5- 
6b). Conversely, we would get a dark pigment if we 
mixed equally high proportions of blue, green, and red 
paint based on subtractive color theory as shown in 
Figure 5-6c. Using three 8-bit images and additive col- 
or theory, we can conceivably display 274 = 16,777,216 
color combinations (Lillesand et al., 2008). For exam- 
ple, RGB brightness values of 255, 255, 0 would yield a 
bright yellow pixel, and RGB brightness values of 


255, 0, 0 would produce a bright red pixel. RGB values 
of 0, 0, 0 yield a black pixel. Grays are produced along 
the gray line in the RGB color coordinate system (Fig- 
ure 5-6a) when equal proportions of blue, green, and 
red are encountered (e.g., an RGB of 127, 127, 127 
produces a medium-gray pixel on the screen or hard- 
copy device). 


Color Look-Up Tables: 8-Bit 


How do we control the exact gray tone or color of the 
pixel on the computer screen after we have extracted a 
byte of remotely sensed data from the mass storage de- 
vice? The gray tone or color of an individual pixel on a 
computer screen is controlled by the size and charac- 
teristics of a separate block of computer memory 
called a color look-up table, which contains the exact 
disposition of each combination of red, green, and blue 
values associated with each 8-bit pixel. Evaluating the 
nature of an 8-bit image processor and associated color 
look-up table (Figure 5-7) provides insight into the way 
the remote sensing brightness values and color look-up 
table interact. Two examples of color look-up tables 
are provided to demonstrate how black-and-white and 
color density-sliced brightness maps are produced. 


In Example | (Figure 5-7), we see that the first 256 ele- 
ments of the look-up table coincide with progressively 
greater values of the red, green, and blue components, 
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FIGURE 5-7 Components of an 8-bit digital image processing system. The image processor display memory is filled with the 
8-bit remote sensing brightness values from a single band of imagery. These values are then manipulated for display by spec- 
ifying the contents of the 256-element color look-up table. An 8-bit digital-to-analog converter (DAC) then converts the dig- 
ital look-up table values into analog signals that are used to modulate the intensity of the red, green, and blue (RGB) guns 
that create the image on a color CRT screen. DACs are not required if a digital monitor is used. Also, most displays are now 
based on liquid crystal display (LCD) technology. 


ranging from 0,0,0, which would be black on the value of 127, the RGB value located at table entry 127 
screen, to 127, 127, 127, which would be mid-gray, and = (with RGB values of 127, 127, 127) would be passed to 
255, 255, 255, which is bright white. Thus, ifa pixelina — the 8-bit DAC converter and a mid-gray pixel would be 
single band of remotely sensed data had a brightness _ displayed on the screen. This type of logic was applied 
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Density Slicing Using an 8-Bit Color Look-Up Table 


a. Landsat Thematic Mapper band 4 image of 
Charleston, SC, obtained on November 9, 1982. 


c. Predawn thermal infrared image of the Savannah 
River obtained on March 28, 1981. 


b. Density slice based on the logic in Table 5-2. 


d. Density slice based on the logic in Table 5-3. 


FIGURE 5-8 a) Black-and-white display of Landsat Thematic Mapper band 4 (0.76 — 0.90 um) 30 x 30 m data of Charleston, 
SC. b) Color density slice using the logic summarized in Table 5-2. c) Black-and-white display of predawn thermal infrared (8.5 
— 13.5 um) imagery of the Savannah River. Each pixel is approximately 2.8 x 2.8 m on the ground. d) Color density slice using 


the logic summarized in Table 5-3. 


to create the black-and-white brightness map of The- 
matic Mapper band 4 data of Charleston, SC, shown 
in Figure 5-8a. This is often referred to as a true 8-bit 
black-and-white display because there is no generaliza- 
tion of the original remote sensor data. This means 


that there is a one-to-one relationship between the 8 
bits of remote sensor input data and the 8-bit color 
look-up table (refer to Figure 5-2a). This is the ideal 
mechanism for displaying remote sensor data. The 
same logic was applied to produce the 8-bit display of 
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TABLE 5-2 Class intervals and color look-up table values for color density slicing the Charleston, SC, Thematic Mapper band 


4 scene shown in Figure 5-8a and b. 


Color Look-up Table Value Brightness Value 
Color Class 

Interval Visual color Red Green Blue Low High 

1 Cyan 0 255 255 0 16 
Shade of gray 17 17 17 17 17 

Shade of gray 18 18 18 18 18 

Shade of gray 19 19 19 19 19 

Shade of gray 58 58 58 58 58 

Shade of gray 59 59 59 59 59 

2 Red 255 0 0 60 255 


TABLE 5-3 Class intervals and color look-up table values for color density slicing the predawn thermal infrared image of the 


thermal plume shown in Figure 5-8c and d. 


Apparent Temperature 
Color Look-up Table Value (°C) Brightness Value 
Visual Low Upper 

Color Class Interval Color Red Green Blue Value Value Low High 
1. Land Gray 127 127 127 -3.0 11.6 0 73, 
2. River ambient Dark blue 0 0 120 ile) 12.2 74 76 
Bh are Blue 0 0 255 12.4 13.0 77 80 
4.1.2-2.8°C Green 0) 255 0 132 14.8 81 89 
5. 3.0 — 5.0°C Yellow 255 25) 0 15.0) 17/0 90 100 
O; 52 = ILE Orange 255 50 0 2 22.0 101 125 
7.10.2 — 20.0°C Red 255 0 0 22.2 32.0 126 176 
8. >20°C White 255 255 255 S72 48.0 177 255 


the predawn thermal infrared image of the Savannah 
River shown in Figure 5-8c. 


Lay people and scientists who analyze remotely sensed 
data are rarely content with the display of the funda- 
mental gray tone (individual band) or color (using 
multiple bands) information contained in a dataset. 
They usually want to highlight in color certain bright- 
ness values in the imagery associated with important 
phenomena (Richards, 2013; Konecny, 2014). The im- 
age analyst can accomplish this task by filling the color 
look-up table with very specific values (Tables 5-2 and 
5-3). 


To demonstrate this point, in Example 2 in Figure 5-7, 
the first entry in the table, 0, is given an RGB value of 
0, 255, 255. Therefore, any pixel in a single band of im- 
agery with a value of 0 would be displayed as cyan (a 
bright blue-green) on the computer screen. In this way 
it is possible to create a special-purpose look-up table 
with those colors in it that are of greatest value to the 
analyst. This is precisely the mechanism by which a 
single band of remote sensor data is color density 
sliced. For example, if we wanted to highlight just the 
water and the most healthy vegetation found within the 
Landsat TM band 4 image of Charleston, SC, we 
could density slice the image shown in Figure 5-8b 
based on the color look-up table values summarized in 


164 INTRODUCTORY DIGITAL IMAGE PROCESSING 


Example 2 (Figure 5-7 and Table 5-2). The color look- 
up table has been modified so that all pixels between 0 
and 16 have RGB values of 0, 255, 255 (cyan). Color 
look-up table values from 60 to 255 have RGB values 
of 255, 0, 0 (red). All values between 17 and 59 have 
the normal grayscale look-up table value; for example, 
a pixel with a BV of 17 has an RGB value 17, 17, 17, 
which will result in a dark gray pixel on the screen. 


Entirely different brightness value class intervals were 
selected to density slice the predawn Savannah River 
thermal infrared image as shown in Figure 5-8d. The 
class intervals and associated color look-up table val- 
ues for the Savannah River color density-sliced image 
are found in Table 5-3. 


Color Look-Up Tables: 24-Bit 


Much greater flexibility is provided when multiple 8-bit 
images can be stored and evaluated all at one time. For 
example, Figure 5-9 depicts the configuration of a 24- 
bit image processing system complete with three 8-bit 
banks of image processor memory and three 8-bit col- 
or look-up tables, one for each image memory bank. 
Thus, three separate 8-bit images could be stored at full 
resolution in the image processor memory banks (one 
in the red, one in the green, and one in the blue). Three 
separate 8-bit DACs continuously read the brightness 
value of a pixel in each of the red, green, and blue im- 
age planes and transform this digital value into an ana- 
log signal that can be used to modulate the intensity of 
the red, green, and blue (RGB) tricolor guns on the 
CRT screen. For example, if pixel (1, 1) in the red im- 
age plane has a brightness value of 255, and pixel (1, 1) 
in both the green and blue image planes have bright- 
ness values of 0, then a bright red pixel (255, 0, 0) 
would be displayed at location (1, 1) on the computer 
screen. More than 16.7 million RGB color combina- 
tions can be produced using this configuration. Obvi- 
ously, this provides a much more ideal palette of colors 
to choose from, all of which can be displayed on the 
computer screen at one time. What we have just de- 
scribed is the fundamental basis behind the creation of 
additive color composites. 


Color Composites 


High spectral (color) resolution is important when pro- 
ducing color composites. For example, if a false-color 
reflective infrared TM image is to be displayed accu- 
rately, each 8-bit input image (TM band 4 = near infra- 
red, TM band 3 = red, and TM band 2 = green) must 
be assigned 8 bits of red, green, and blue image proces- 
sor memory, respectively. In this case, the 24-bit system 
with three 8-bit color look-up tables would provide a 
true-color rendition of each pixel on the screen, as pre- 


viously discussed. This is true additive color combining 
with no generalization taking place. 


Additive color composites produced from various 
Landsat TM band combinations of February 3, 1994, 
data are presented in Figure 5-10. The first is a natural 
color composite in which TM bands 3, 2, and | are 
placed in the red, green, and blue image processor dis- 
play memory planes, respectively (Figure 5-10a). This 
is what the terrain would look like if the analyst were 
onboard the satellite platform looking down on South 
Carolina. 


A color-infrared color composite of TM bands 4, 3, 
and 2 (RGB) is displayed in Figure 5-10b. Healthy veg- 
etation shows up in shades of red because photosyn- 
thesizing vegetation absorbs most of the green and red 
incident energy but reflects approximately half of the 
incident near-infrared energy (discussed in Chapter 8). 
Dense urban areas reflect approximately equal propor- 
tions of near-infrared, red, and green energy; therefore 
they appear as steel gray. Moist wetland areas appear 
in shades of greenish brown. The third color composite 
was produced using bands 4 (near-infrared), 5 (middle- 
infrared), and 3 (red) (Figure 5-10c). The composite 
provides good definition of the land—water interface. 
Vegetation type and condition appear in shades of 
brown, green, and orange. The more moist the soil, the 
darker it appears. Figure 5-10d was created using TM 
bands 7, 4, and 2 (RGB). Many analysts like this com- 
bination because vegetation is presented in familiar 
green tones. Also, the mid-infrared TM band 7 helps 
discriminate moisture content in both vegetation and 
soils. Urban areas appear in varying shades of magen- 
ta. Dark green areas correspond to upland forest, 
while greenish brown areas are wetland. But what 
about all the other three-band color composites that 
can be produced from the same Landsat TM data? 


Optimum Index Factor 

Chavez et al. (1984) developed an optimum index factor 
(OIF) that ranks the 20 three-band combinations that 
can be made from six bands of TM data (not including 
the thermal-infrared band). The technique, however, is 
applicable to any multispectral remote sensing dataset. 
It is based on the amount of total variance and correla- 
tion within and between various band combinations. 
The algorithm used to compute the OIF for any subset 
of three bands is: 


(5.1) 
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Components of a 24-Bit Digital Image Processing System 
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FIGURE 5-9 Components of a 24-bit digital image processing system. The image processor can store and continuously eval- 
uate up to three 8-bit remotely sensed images. Three 8-bit digital-to-analog converters (DACs) scan the contents of the three 
8-bit color look-up tables. The pixel color displayed on the screen will be just one possible combination (e.g., red = 255, 0, 0) 
out of a possible 16,777,216 colors that can be displayed at any time using the 24-bit image processing system. DACs are not 


required if a digital monitor is used. 


where s; is the standard deviation for band k, and r;is 
the absolute value of the correlation coefficient be- 
tween any two of the three bands being evaluated. The 
three-band combination with the largest OIF generally 
has the most information (measured by variance) with 
the least amount of duplication (measured by correla- 


tion). Combinations within two or three rankings of 
each other produce similar results. 


Application of the OIF criteria to the 1982 Charleston, 
SC, Landsat TM dataset (excluding the thermal band) 
resulted in 20 combinations (Table 5-4). The standard 
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Color Composites of Landsat Thematic Mapper Data 
of Charleston, SC, Collected on February 3, 1994 


c. Landsat TM bands 4, 5, 3 = RGB. d. Landsat TM bands 7, 4, 2 = RGB. 


FIGURE 5-10 Color composites of Landsat Thematic Mapper data of Charleston, SC, collected on February 3, 1994. 

a) Composite of Landsat TM bands 3, 2, and 1 placed in the red, green, and blue (RGB) image processor memory planes, 
respectively. b) TM bands 4, 3, and 2 = RGB. c) TM bands 4, 5, 3 = RGB. d) TM bands 7, 4, 2 = RGB. Original imagery courtesy 
of NASA. 
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TABLE 5-4 Optimum Index Factors for six of the Charles- 
ton, SC, Landsat Thematic Mapper bands. 


Rank Combination OIF 

1 1h4y5 ZY NS 
2 2,4,5 23.549 
5 3,4,5 22.599 
4 psy 7 20.785 
5 Unt 2 20.460 
6 4,5,7 20.100 
7 lReSaS 19.009 
8 (25 18.657 
9. 1,3,4 18.241 
10 2,3), 18.164 
11 2, 3, 3 17.920 
12 Sy 7/ 17.840 
ils) Wao 17.682 
14 2,4,7 SWZ 
15 3,4,7 17.141 
16 2,3, 15.492 
W7/ ney W 1227/1 
18 Nl, 2, 7 11.615 
19 PST 10.387 
20 In a & 8.428 
Six bands combined three at a time allows 20 combina- 

tions. The thermal infrared band 6 (10.4 to 12.5 um) was 

not used. 
For example, the OIF for Landsat TM band combination 1, 

4, and 5 from Table 4-7 is: 

“oag F056 r088 ~ 27-137. 


deviations and between-band correlation coefficients 
were obtained from Table 4-7. A three-band combina- 
tion using bands 1, 4, and 5 should provide the opti- 
mum color composite, with bands 2, 4, and 5 and 3, 4, 
and 5 just about as good. Generally, the best three- 
band combinations include one of the visible bands 
(TM 1, 2, or 3), and one of the longer-wavelength in- 
frared bands (TM 5 or 7), along with TM band 4. TM 
band 4 was present in five of the first six rankings. Such 
information can be used to select the most useful 
bands for three-band color composites. The analyst 


must then decide what color to assign each band (red, 
green, or blue) in the color composite. 


Sheffield Index 

Sheffield (1985) developed a statistical band selection 
index based on the size of the hyperspace spanned by 
the three bands under investigation. Sheffield suggest- 
ed that the bands with the largest hypervolumes be se- 
lected. The index is based on computing the 
determinant of each p by p sub-matrix generated from 
the original 6 X 6 covariance matrix (if six bands are 
under investigation). The Sheffield Index (SI) is: 


SI = |Co (5.2) 


“px p 


where |Cov, . | is the determinant of the covariance 
matrix of subset size p. In this case, p = 3 because we 
are trying to discover the optimum three-band combi- 
nation for image display purposes. In effect, the SI is 
first computed from a 3 X 3 covariance matrix derived 
from just band 1, 2, and 3 data. It is then computed 
from a covariance matrix derived from just band 1, 2, 
and 4 data, etc. This process continues for all 20 possi- 
ble band combinations if six bands are under investiga- 
tion, as in the previous example. The band 
combination that results in the largest determinant is 
selected for image display. All of the information nec- 
essary to compute the SI is actually present in the orig- 
inal 6 X 6 covariance matrix. The Sheffield Index can 
be extended to datasets containing n bands. Beauche- 
min and Fung (2001) suggested using a normalized 
version of the Sheffield Index. 


Independent Component Analysis-Based (ICA- 
Based) Fusion for Color Display of 
Hyperspectral Images 


Hyperspectral imagery often consists of many hun- 
dreds of bands. It is difficult to know which three of 
the bands to display on a computer screen. Zhu et al. 
(2011) developed an ICA-based dimensionality reduc- 
tion method to fuse the n hyperspectral images to three 
independent component images for color display. They 
suggest it performs better than standard principal 
components analysis (PCA) dimensionality reduction 
for hyperspectral data display. 


Merging (Fusing) Remotely 
Ls Sensed Data 


Image analysts often merge (fuse) different types of re- 
mote sensor data such as: 
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SPOT Imagery of Marco Island, FL 


a. Band 1 (0.50 — 0.59 um) at 20 x 20 m. 


c. Band 3 (0.79 — 0.89 um) at 20 x 20 m. 


b. Band 2 (0.61 — 0.68 um) at 20 x 20 m. 


ou : o* .* . Ary 


d. Panchromatic (0.51 — 0.73 um) at 10 x 10 m. 


FIGURE 5-11 Individual bands of SPOT 20 x 20 multispectral and 10 x 10 m panchromatic imagery used to create the nor- 
mal color-infrared color composites in Figure 5-12a and the merged (fused) color-infrared color composite in Figure 5-12b 
[images courtesy of CNES (www.cnes.fr). Protected information. All rights reserved © CNES (2014)]. 


SPOT 10 x 10 m PAN data with Landsat Thematic 
Mapper 30 x 30 m data (e.g., Chavez and Bowell, 
1988), 

Multispectral data (e.g., SPOT XS, Landsat TM, 
IKONOS) with active microwave (radar) and/or 
other data (e.g., Chen et al., 2003; Klonus and 
Ehlers, 2009), 

IKONOS 1 x 1 m data with Landsat Enhanced The- 
matic Mapper (ETM) 30 x 30 m data (Chen et al., 
2011), 


Landsat ETM* multispectral 30 x 30 m data with 
ETM™ 15 x15 m panchromatic data (Park and 
Kang, 2004), 

QuickBird 0.7 x 0.7 m panchromatic with 2.8 x 2.8 
m multispectral data (Aiazzi et al., 2007), 
Hyperspectral data into three independent compo- 
nents for visual display (Zhu et al., 2011), and 
CARTOSAT-1 panchromatic 2.5 x 2.5 m data and 
IRS-P6 LISS-IV 5.8 x 5.8 m multispectral data 
(Jalan and Sokhi, 2012). 
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When lower spatial resolution imagery such as 30 x 30 
m Landsat Thematic Mapper data are fused (merged) 
with higher spatial resolution panchromatic imagery 
such as | X 1 m IKONOS panchromatic data, the pro- 
cess is commonly referred to as pan-sharpening. The 
goal of pan-sharpening is to render a sharpened image 
incorporating the full spatial information content of 
the panchromatic image without introducing spectral 
distortions to the multispectral input data (Jalan and 
Sokhi, 2012). 


Merging (fusing) remotely sensed data obtained using 
different remote sensors must be performed carefully. 
All datasets to be merged must be accurately registered 
to one another and resampled (Chapter 6) to the same 
pixel size. Several component-substitution (CS) meth- 
ods exist for merging (fusing) the data sets, including 
(Park and Kang, 2004; Schowengerdt, 2006; Aiazzi et 
al., 2007; Alparone et al., 2007; Klonus and Ehlers, 
2009; Prost, 2013): 


¢ simple band-substitution methods; 


* color space transformation and component substi- 
tution methods using RGB, intensity—hue-satura- 
tion, and chromaticity color coordinate systems; 


¢ Principal Component Analysis (PCA), Indepen- 
dent Component Analysis (ICA) or Gram-Schmidt 
substitution; 

* pixel-by-pixel addition of high-frequency informa- 
tion to a lower spatial resolution dataset; 

* image fusion based on regression kriging; and 


* smoothing filter-based intensity modulation image 
fusion. 


There are other pan-sharpening algorithms available 
such as multi-resolution analysis (MRA) based on the 
use of wavelets or Laplacian pyramids and Ehler’s fu- 
sion method based on an IHS transform and Fourier 
filtering that are not discussed in this introductory text 
(e.g., Alparone et al., 2007; Klonus and Ehlers, 2007). 


Simple Band Substitution 


Simple band substitution is one of the most heavily 
used image fusion methods. For example, individual 
SPOT bands of Marco Island, FL, are displayed in 
Figure 5-11. The data were geometrically rectified to a 
Universal Transverse Mercator (UTM) projection and 
then resampled to 10 x 10 m pixels using bilinear inter- 
polation. Note how the 20 x 20 m multispectral bands 
(green, red, and near-infrared) appear out of focus. 
This is typical of 20 x 20 m data of urban areas. Con- 
versely, the 10 x 10 m panchromatic data exhibits more 
detailed road network information and perimeter de- 
tail for some of the larger buildings. Merging (fusion) 
of the 20 x 20 m multispectral data with the 10 x 10m 
panchromatic data could be of value. 


Figure 5-12a is a color-infrared color composite with 
the band 3 (near-infrared), band 2 (red), and band 1 
(green) imagery placed in the red, green, and blue im- 
age processor display memory banks, respectively (i.e., 
bands 3, 2, 1 = RGB). The SPOT panchromatic data 
span the spectral region from 0.51 to 0.73 um. There- 
fore, it is a record of both green and red energy. It can 
be substituted directly for either the green (SPOT 1) or 
red (SPOT 2) bands. Figure 5-12b is a display of the 
merged (fused) dataset with SPOT 3 (near-infrared), 
SPOT panchromatic, and SPOT | (green) in the RGB 
image processor memory planes, respectively. The re- 
sult is a display that contains the spatial detail of the 
SPOT panchromatic data (10 < 10 m) and spectral de- 
tail of the 20 x 20 m SPOT multispectral data. This 
method has the advantage of not changing the radio- 
metric qualities of any of the SPOT data. 


Color Space Transformation and 
Component Substitution 


All remotely sensed data presented thus far have been 
in the RGB color coordinate system. Other color coor- 
dinate systems may be of value when presenting re- 
motely sensed data for visual analysis, and some of 
these may be used when different types of remotely 
sensed data are merged. Two frequently used methods 
are the RGB to intensity—hue-saturation (IHS) trans- 
formation and the use of chromaticity coordinates. 


RGB to IHS Transformation and Back Again 

The intensity—hue-saturation (IHS) color coordinate 
system is based on a hypothetical color sphere (Figure 
5-13a). The vertical axis represents intensity (1) which 
varies from black (0) to white (255) and is not associat- 
ed with any color. The circumference of the sphere rep- 
resents hue (H), which is the dominant wavelength of 
color. Hue values begin with 0 at the midpoint of red 
tones and increase counterclockwise around the cir- 
cumference of the sphere to conclude with 255 adja- 
cent to 0. Saturation (S) represents the purity of the 
color and ranges from 0 at the center of the color 
sphere to 255 at the circumference. A saturation of 0 
represents a completely impure color in which all wave- 
lengths are equally represented and which the eye will 
perceive as a shade of gray that ranges from white to 
black depending on intensity. Intermediate values of 
saturation represent pastel shades, whereas high values 
represent more pure, intense colors. All values used in 
this example are scaled to 8 bits, corresponding to 
most digital remote sensor data. Figure 5-13a high- 
lights the location of a single pixel with IHS coordi- 
nates of 190, 0, 220 (i.¢., a relatively intense [190], 
highly saturated [220], red [0] pixel). 


Any RGB multispectral dataset consisting of three 
bands may be transformed into IHS color coordinate 
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Merging (Fusion) of SPOT 20 x 20 m Multispectral 
and 10 x 10 m Panchromatic Data of Marco Island, FL 


a. Color-infrared color composite 
of SPOT band 3 (near-infrared), 
2 (red), and 1 (green) = RGB. 
Each band is 20 x 20 m. 


b. Fused color-infrared color 
composite of SPOT band 3 
(near-infrared), 4 (panchromatic), 
and | (green) = RGB. The 
panchromatic band is 10 x 10 m. 
The composite was created by 
substituting the panchromatic 
band for band 2 (red). 


FIGURE 5-12 a) Merging (fusion) of SPOT multispectral data (20 x 20 m) with SPOT panchromatic data (10 x 10 m) using 
the band substitution method. The 20 x 20 m multispectral data were resampled to 10 x 10 m [images courtesy of CNES 
(www.cnes.fr). Protected information. All rights reserved © CNES (2014)]. 


space using an IHS transformation. This transforma- 
tion is actually a limitation because many remote sens- 
ing datasets contain more than three bands. The 
relationship between the RGB and IHS systems is 
shown diagrammatically in Figure 5-13b. Numerical 
values may be extracted from this diagram for express- 


ing either system in terms of the other. The circle repre- 
sents a horizontal section through the equatorial plane 
of the IHS sphere (Figure 5-13a) with the intensity axis 
passing vertically through the plane of the diagram. 
The corners of the equilateral triangle are located at 
the position of the red, green, and blue hues. Hue 
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Intensity-Hue-Saturation (IHS) 
Color Coordinate System 
255 


Relationship between the RGB and IHS Color Systems 


Green 
H=1 


Red 
H=0 


b. 


FIGURE 5-13 a) Intensity-hue-saturation (IHS) color coordinate system. The primary colors are on the interior of the circle. 
The secondary colors are on the periphery of the circle. b) Relationship between the intensity—-hue-saturation (IHS) color coor- 


dinate system and the RGB color coordinate system. 


changes in a counterclockwise direction around the tri- 
angle, from red (H = 0), to green (H = 1), to blue (H = 
2), and again to red (H = 3). Values of saturation are 0 
at the center of the triangle and increase to a maximum 
of | at the corners. Any perceived color is described by 
a unique set of IHS values. The IHS values can be de- 
rived from the RGB values through transformation 
equations (Sabins, 2007): 


I= R+G+B, (5.3) 
_ G-B 
H= 7-3R° (5.4) 
and 
. §-38 
S= + (5.5) 


for the interval 0 < H < 1, extended to | < H <3. Pel- 
lemans et al. (1993) used different equations to com- 
pute intensity, hue, and saturation for a SPOT dataset 
consisting of three bands of remotely sensed data 
(B Vi, BV, and B V3): 


BV, + BV>+ BY, 


5 ; (5.6) 


Intensity = 


2BV-BY-BY3 


Hue = 
3(BV, -BV3) 


Cc (5.7) 


arctan 


C= 0, if BV, 2 BV, 
where 4 , 
te = nm, if BV, < BV; 


and 


Saturation = 


{6(av,2+Bv,2+BY2-BV,BV,-BV,BV;—BV,BV,) 
S58) 
3 

So what is the benefit of performing an IHS transfor- 
mation? First, it may be used to improve the interpret- 
ability of multispectral color composites. When any 
three spectral bands of multispectral data are com- 
bined in the RGB system, the color composite image 
often lacks saturation, even when the bands have been 
contrast stretched. Therefore, some analysts perform 
an RGB-to-IHS transformation, contrast stretch the 
resultant saturation image, and then convert the IHS 
images back into RGB images using the inverse of the 
equations just presented. The result is usually an im- 
proved color composite. 


The IHS transformation is also often used to merge 
(fuse) lower spatial resolution imagery such as 30 x 30 
m Landsat TM data with higher spatial resolution data 
such as IKONOS panchromatic | x | m imagery (e.g., 
Gonzalez-Audicana, 2004; Park and Kang, 2004; 
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Schowengerdt, 2006; Aiazzi et al., 2007; Yao and Han, 
2010). The method generally involves four steps: 


1. RGB to IHS: Three bands of lower-spatial- 
resolution remote sensor data in RGB color space 
are transformed into three bands in IHS color 
space. 

2. Contrast manipulation: The high-spatial-resolution 
image (e.g., SPOT PAN data or digitized aerial 
photography) is contrast stretched so that it has 
approximately the same variance and mean as the 
intensity (I) image. 

3. Substitution: The stretched, high-spatial-resolution 
image is substituted for the intensity (I) image. 

4. IHS to RGB: The modified IHS dataset is 
transformed back into RGB color space using an 
inverse IHS transformation. The justification for 
replacing the intensity (1) component with the 
stretched higher-spatial-resolution image is that 
the two images have approximately the same 
spectral characteristics 


Ehlers et al. (1990) used this methodology to merge 
SPOT 20 x 20 m multispectral and SPOT panchromat- 
ic 10 x 10 m data. The resulting multiresolution image 
retained the spatial resolution of the 10 x 10 m SPOT 
panchromatic data, yet provided the spectral charac- 
teristics (hue and saturation values) of the SPOT multi- 
spectral data. The enhanced detail available from 
merged images was found to be important for visual 
land-use interpretation and urban growth delineation 
(Ehlers et al., 1990). In a similar study, Carper et al. 
(1990) found that direct substitution of the panchro- 
matic data for intensity (I) derived from the multispec- 
tral data was not ideal for visual interpretation of 
agricultural, forested, or heavily vegetated areas. They 
suggested that the original intensity value obtained in 
step 1 be computed using a weighted average (WA) of 
the SPOT panchromatic and SPOT multispectral data; 
that is, WA = {[(2 x SPOT Pan) + SPOT XS3]/3}. 


Chavez et al. (1991) cautioned that of all the methods 
used to merge multi-resolution data, the IHS method 
distorts the spectral characteristics the most and 
should be used with caution if detailed radiometric 
analysis of the data is to be performed. Fortunately, 
numerous scientists have improved upon the IHS sub- 
stitution method. For example, Park and Kang (2004) 
developed a spatially adaptive image fusion algorithm 
that controls the amount of high frequency informa- 
tion extracted from the high spatial resolution image 
that is inserted into the resampled multispectral data. 
Choi (2006) introduced an IHS fusion pan-sharpening 
method. Aiazzi et al. (2007) used multivariate regres- 
sion to retain the quality of the spectral information 
without sacrificing spatial detail. 


FIGURE 5-14 Chromaticity color coordinate system. 


Chromaticity Color Coordinate System and the 
Brovey Transformation 

A chromaticity color coordinate system can be used to 
specify color (Ready, 2013). A chromaticity diagram is 
shown in Figure 5-14. The coordinates in the chroma- 
ticity diagram represent the relative fractions of each 
of the primary colors (red, green, and blue) present in a 
given color. Since the sum of all three primaries must 
add to 1, we have the relationship: 


R+G+B=l (5.9) 


or 
B=1-(R+@O). (5.10) 


Entry into the chromaticity diagram is made using the 
following relationships: 


_ R 
a REGSEB’ ei) 
7 G 
y R+GHB’ eal2) 
and 
= B 
a R+G+B’ G9) 


where R, G, and B represent the amounts of red, green, 
and blue needed to form any particular color, and x, y, 
and z represent the corresponding normalized color 
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components, also known as trichromatic coefficients 
(Kulkarni, 2001). Only x and y are required to specify 
the chromaticity coordinates of a color in the diagram 
since x + y + z = 1. For example, the point marked 
“green” in Figure 5-14 has 70% green and 10% red 
content. It follows from Equation 5.10 that the compo- 
sition of blue is 20%. 


The human eye responds to light only in the range 380 
to 800 nm (Myler, 2013). Therefore, the positions of 
the various spectrum colors from 380 to 800 nm are in- 
dicated around the boundary of the chromaticity dia- 
gram. These are the pure colors. Any point not actually 
on the boundary but within the diagram represents 
some mixture of spectrum colors. The point of equal 
energy shown in Figure 5-14 corresponds to equal frac- 
tions of the three primary colors and represents the 
CIE standard for white light. Any point located on the 
boundary of the chromaticity diagram is said to be 
completely saturated. As a point leaves the boundary 
and approaches the point of equal energy, more white 
light is added to the color and it becomes less saturat- 
ed. The chromaticity diagram is useful for color mix- 
ing, because a straight-line segment joining any two 
points in the diagram defines all of the color variations 
that can be formed by combining the two colors addi- 
tively. 


The Brovey transform may be used to merge (fuse) im- 
ages with different spatial and spectral characteristics. 
It is based on the chromaticity transform and is a much 
simpler technique than the RGB-to-IHS transforma- 
tion. The Brovey transform also can be applied to indi- 
vidual bands if desired. It is based on the following 
intensity modulation (Liu, 2000a): 


RxP 
Redprovey => (5.14) 
Greeng,ovey = so (5.15) 
BxP 
Blue gyoyey => (5.16) 
and 
c= ee (5.17) 


where R, G, and B are the spectral band images of in- 
terest (e.g., 30 x 30 m Landsat ETM‘ bands 4, 3, and 
2) to be placed in the red, green, and blue image pro- 
cessor memory planes, respectively, P is a coregistered 
band of higher spatial resolution data (e.g., 1 x 1 m 
IKONOS panchromatic data), and J = intensity. An 
example of the Brovey transform used to merge IKO- 
NOS 4 Xx 4 m multispectral data (bands 4, 3, and 2) 


with IKONOS | x 1 m panchromatic data is shown in 
Figure 5-15. The merged (fused) dataset now has the 
spectral characteristics of the multispectral data and 
the spatial characteristics of the high-resolution pan- 
chromatic data. 


Both the RGB-to-IHS transformation and the Brovey 
transform can cause color distortion if the spectral 
range of the intensity replacement (or modulation) im- 
age (i.e., the panchromatic band) is different from the 
spectral range of the three lower-resolution bands. The 
Brovey transform was developed to visually increase 
contrast in the low and high ends of an image’s histo- 
gram (i.e., to provide contrast in shadows, water, and 
high-reflectance areas such as urban features). Conse- 
quently, the Brovey transform should not be used if 
preserving the original scene radiometry is important. 
However, it is good for producing RGB images with a 
higher degree of contrast in the low and high ends of 
the image histogram and for producing visually ap- 
pealing images. 


Principal Component Analysis (PCA), 
Independent Component Analysis 
(ICA), or Gram-Schmidt Substitution 


PCA is a commonly used method for spectral transfor- 
mation of the original remote sensor data to produce 
uncorrelated principal component images (Pratt, 2013) 
(PCA is discussed in detail in Chapter 8). PCA-based 
pan-sharpening methods make the assumption that the 
first principal component image (e.g., PC1) explains 
the largest amount of variance in the entire original re- 
mote sensing dataset (Shah et al., 2008). Therefore, 
PC] is an excellent choice to be replaced with the high 
spatial resolution image, usually a panchromatic im- 
age. An inverse principal component analysis is then 
applied to the substituted PCl image along with the 
other principal component images to obtain a high- 
resolution pan-sharpened image. The panchromatic 
image is usually histogram-matched with the first PC1 
image before the substitution takes place (Campbell, 
2007). For example, Chavez et al. (1991) used PCA ap- 
plied to six 30 x 30 m Landsat TM bands. The SPOT 
panchromatic 10 x 10 m data were contrast stretched 
to have approximately the same variance and average 
as the first principal component image (PCl). The 
stretched panchromatic data were substituted for the 
first principal component image and the data were 
transformed back into RGB space. The advantage of 
PCA-based pan sharpening is that the number of 
bands is not restricted (such as for the original IHS or 
Brovey fusion methods) (Klonus and Ehlers, 2009). 
Shah et al. (2008) and Yang and Gong (2012) devel- 
oped improved PCA substitution pan-sharpening 
methods. 
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IKONOS Multispectral and Panchromatic Imagery of Columbia, SC 


c. Band 4 (0.76 — 0.90 um) 4 x 4 m. d. Panchromatic band (0.45 — 0.90 um) 1 x 1 m. 


V4 


e. Color composite (RGB = bands 4, 3, 2). f. Brovey transform of bands 4, 3, 2 and panchromatic. 


FIGURE 5-15 IKONOS imagery of a business park in Columbia, SC. a-d) Individual 4 x 4 m multispectral bands and the 1 x 1 
m panchromatic band are displayed. e) Standard color composite of IKONOS bands 4, 3, and 2. f) Color composite of a 
merged dataset created using a Brovey transform (images courtesy of DigitalGlobe, Inc.). 
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Chen et al. (2011) used independent component analy- 
sis (ICA) to separate the main body, spectral detail, 
and spatial detail in a Landsat ETM™ 30 x 30 m multi- 
spectral dataset. They then substituted the information 
from an IKONOS high spatial resolution 1 x 1 m pan- 
chromatic image for the main body independent com- 
ponent data. After inversion back to RGB space, the 
new data (image) contained the spatial information of 
the high-resolution image and the spectral characteris- 
tics of the original multispectral image. 


The Gram-Schmidt image fusion method first synthe- 
sizes a panchromatic band from the lower spatial reso- 
lution spectral bands (Klonus and Ehlers, 2009). This 
is achieved by averaging the multispectral bands. A 
Gram-Schmidt transformation is then performed for 
the simulated panchromatic band and the multispec- 
tral bands with the simulated panchromatic band em- 
ployed as the first band in the dataset. Then the high 
spatial resolution panchromatic band replaces the first 
Gram-Schmidt band. Finally, an inverse Gram- 
Schmidt transformation is performed to create the 
pan-sharpened multispectral bands in RGB color 
space (Laban et al., 2000). Klonus and Ehler (2009) 
suggest that the method produces good results for fus- 
ing images from one sensor, but because it is a statisti- 
cal procedure like principal components analysis 
(PCA), the fusion results may vary depending on the 
selected image subsets. Gram-Schmidt image fusion is 
available in ENVI software. 


Pixel-by-Pixel Addition of High- 
Frequency Information 


Schowengerdt (1980) introduced the concept of utiliz- 
ing image-derived high-frequency information in the 
fusion process. Chavez (1986), Chavez and Bowell 
(1988), and Chavez et al. (1991) merged both digitized 
National High Altitude Program photography and 
SPOT panchromatic data with Landsat Thematic 
Mapper data using a high-pass spatial filter applied to 
the high-spatial-resolution imagery. The resultant 
high-pass image contains high-frequency information 
that is related mostly to spatial characteristics of the 
scene. The spatial filter removes most of the spectral in- 
formation. The high-pass filter results were added, pix- 
el by pixel, to the lower-spatial-resolution TM data. 
This process merged the spatial information of the 
higher-spatial-resolution dataset with the higher spec- 
tral resolution inherent in the TM dataset. Chavez et 
al. (1991) found that this multisensor fusion technique 
distorted the spectral characteristics the least. 


Gonzalez-Audicana et al. (2004) improved upon the 
use of specially-prepared high-pass filtered data by us- 
ing wavelet decomposition. Jalan and Sokhi (2012) 
compared the high-pass filtering (HPF) pan-sharpen- 


ing fusion method with four other image fusion meth- 
ods. The HPF method was capable of providing fusion 
results with high spectral integrity as well as good spa- 
tial enhancement when applied to CARTOSAT-1 pan- 
chromatic data (2.5 <x 2.5 m) and IRS-P6 LISS-IV 
multispectral imagery (5.8 X 5.8 m). 


Fusion based on Regression Kriging 


Geostatistics and kriging were introduced in Chapter 
4. Meng et al. (2010) developed an image fusion meth- 
od based on regression kriging which takes into con- 
sideration the correlation between the image to be 
fused and the image with finer spatial resolution, the 
spatial autocorrelation among pixels in the high spatial 
resolution image, and the unbiased estimation with 
minimized variance. Qualitative assessments suggest 
that there is no apparent color distortion in the fused 
images. Van der Meer (2012) reviews other geostatisti- 
cal image fusion research. 


Smoothing Filter-Based Intensity 
Modulation Image Fusion 


Liu (2000ab) developed a Smoothing Filter-based In- 
tensity Modulation (SFIM) image fusion technique 
based on the algorithm: 


SFL BV (5.18) 


mean 


where BV;,,, is a pixel from the low spatial resolution 
image, BVp;g; is a pixel from the high spatial resolution 
image, and BV,,.qn 1S a simulated low resolution pixel 
derived from the high-resolution image using an aver- 
aging filter for a neighborhood equivalent in size to the 
spatial resolution of the low-resolution data. For ex- 
ample, suppose the high-resolution image consisted of 
SPOT 10 x 10 m panchromatic data and the low-reso- 
lution image consisted of Landsat ETM* 30 x 30 m 
data. In this case, the BV,,,.,,, value would be the aver- 
age of the nine 10 x 10 m pixels centered on the pixel 
under investigation in the high spatial resolution datas- 
et. Liu (2000a) suggests that the SFIM can produce 
optimally fused data without altering the spectral 
properties of the original image if the coregistration er- 
ror is minimal. 


Liu et al. (2012) undertook an objective assessment of 
various multiresolution image fusion algorithms. 
Khaleghi et al. (2013) provided a taxonomy of data fu- 
sion methodologies. 
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Euclidean Distance Measurement 
Pythagorean Theorem 
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FIGURE 5-16 a) The calculation of the Euclidean distance between two points in an X,Y Cartesian coordinate system using 
the Pythagorean theorem. b) The calculation of the Manhattan distance between Point #1 and Point #2 in an X, Y Cartesian 


coordinate system. 


Length (Distance) 
Measurement 


Sometimes while viewing an image on the computer 
screen, an analyst desires to measure the length of an 
object, its area, or its shape. Most digital image pro- 
cessing programs have graphical user interfaces that al- 
low the analyst to first identify the point, line, or area 
feature in the imagery and then extract the type of 
measurement desired. 


Linear Distance Measurement Based 
on the Pythagorean Theorem 


One of the most common measurements you will need 
to calculate is the Euclidean distance between two 
points, which is easily computed using the Pythagore- 
an theorem. The Pythagorean theorem is based on the 
relationship between the three sides of a right triangle 
and is typically stated as follows: In any right triangle 
(a triangle where one inside angle = 90° ), the length of 
the line segment opposite the right angle (the hypote- 
nuse) is equal to the square root of the squared sums of 
the lengths of the other two legs (1.e., the two sides oth- 
er than the hypotenuse). These relationships are shown 
in Figure 5-16a. 


In mathematical terms, the equation is expressed as: 


C= A+B (5.19) 
where A and B are the lengths of two line segments 
that are not the hypotenuse (C). Therefore, if you 
know the projected X,Y coordinates for any two 
points, it is a relatively straightforward matter to deter- 


TABLE 5-5 The coordinates of Point #1 and Point #2 in Fig- 
ure 5-16 in meters. 


Point X-coordinate Y-coordinate 
1 D 2 
2 7 5 


mine the distance between them—you simply deter- 
mine the lengths of the two legs and then calculate the 
length of the hypotenuse. For example, suppose you 
want to measure the distance between Point #1 and 
Point #2 in the Cartesian coordinate system shown in 
Figure 5-16a and listed in Table 5-5. To determine the 
distance between these two points it is first necessary to 
calculate the distance of the two sides of the triangle 
that form the right angle. To determine the length of 
line A, one of the Y values is subtracted from the other 
Y value (2 — 5 = —3). To determine length of line B, one 
of the X values is subtracted from the other X¥ value (2 
—7=-5). Both values are used in Equation 5.19: 


C= (3) +5)" 


C= 9425 
C= 34 
C= 34 
C = 5.83m 


The distance between Point #1 and Point #2 in Figure 
5-16a is 5.83 m. 
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Euclidian Distance Measurement 


95m 


FIGURE 5-17 Euclidean distance measurements are based on the Pythagorean theorem where the lengths of lines A and B 
in a right triangle can be used to calculate the length of the hypotenuse, C. In this example, the distance from a point in front 
of the single-family residence to a corner of the adjacent field is calculated (aerial photography courtesy state of Utah). 


TABLE 5-6 The coordinates of Point #1 and Point #2 in Fig- 
ure 5-17 in a Universal Transverse Mercator (UTM) Cartesian 
coordinate system (Zone 12 N). 


Point UTM UTM 
X-coordinate Y-coordinate 
1 432,860 4,426,841 
2 432,966 4,427,036 


Normally, the coordinates of the points that you are in- 
vestigating are described in terms of their location in a 
particular coordinate system, e.g., the Universal Trans- 
verse Mercator (UTM) map projection. For example, 
suppose you wanted to compute the distance between 
Point #1 and Point #2 in the agricultural field shown 
in Figure 5-17. The coordinates of the two points are 
listed in Table 5-6. 


In this case the computation would be: 
Ce = (432860 — 432966) + (4426841 — 4427036) 
C= 106) sy 


C? = 11236 + 38025 


C? = 49261 
C = /49261 
C= 22195 m 


A limitation of using the Pythagorean theorem is that 
this method only works when the points used in the 
measurement are comparable—that is, they are in a co- 
ordinate system based on a projection such as UTM. 
Also, this method will not work for latitude and longi- 
tude or for projected points that are great distances 
from one another because of the curvature of the 
Earth. 


Manhattan Distance Measurement 


Linear distance measurement based on the Pythagore- 
an theorem is useful for many applications, but it does 
have limitations. One limitation is the fact that to get 
from Point #1 to Point #2 in an urban environment— 
and even in many natural settings—it may not be logi- 
cal to measure the Euclidean distance between two 
points using a straight line (i.e., using the hypotenuse) 
“as the crow flies.” To remedy this, the Manhattan dis- 
tance between two points may be calculated, where: 


Manhattan 4: tance ne -X,| + a - Y,| (5.20) 
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Euclidean Distance versus Manhattan Distance Measurement 
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FIGURE 5-18 The white line from the intersection at location A to the intersection at location B would require you to walk 
through commercial buildings, climb over fences, and pass through parks and natural areas. Therefore, the Euclidean distance 
(white line) would not be representative of the practical distance to walk from location A to location B. Conversely, it is possi- 
ble to determine the Manhattan “round-the-block” distance for the same journey. In this example, all of the lines except for 
the white line, have exactly the same length from location A to location B as they traverse through the city (aerial photogra- 


phy courtesy of state of Utah). 


The Manhattan distance (sometimes referred to as 
“round-the-block” or “city block” distance) between 
two points utilizes the lengths of the two sides of the 
right triangle, but not the hypotenuse (Figure 5-16b). 
This is analogous to going from Point #1 to Point #2 
in a city where you cannot simply walk through build- 
ings or climb over fences. Rather, you are required to 
walk around the block to get from Point #1 to Point 
#2. For example, the Manhattan distance between 
Point #1 and Point #2 in Figure 5-16b is: 


Manhattan a: tance [2-714 12-5 


Manhattan 4: tance 5+3 


Manhattan 4: ance 8m 


Note that this is not the same value (5.83 m) calculated 
using the Pythagorean theorem in the previous exam- 
ple (Figure 5-16a). 


Therefore, while it is often useful to determine the Eu- 
clidean distance between Point #1 and Point #2 “as the 
crow flies” using the Pythagorean theorem, other fac- 
tors must sometimes be considered before a practical 
distance measurement can be made. In fact, many geo- 
graphic areas have features that impede making direct 
Pythagorean theorem-based distance measurements 
between two points. 


For example, suppose we need to go from location A to 
location B in downtown Salt Lake City, UT (Figure 5- 
18). We are interested in traveling the shortest distance 
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Calculating the Perimeter and Area of a Complex Polygon 


FIGURE 5-19 A six-sided polygon circumscribes a subdivision located in Spanish Fork, UT (vertex #1 is used twice to close 
the polygon). The perimeter and area of this polygon can be calculated using Equations 5.21 and 5.22, respectively. The 
length of the longest diagonal that can be inscribed in the polygon is 390.48 m based on the Pythagorean theorem. The area 
and longest diagonal information can be used to compute the compactness of the polygon (aerial photography courtesy 


state of Utah). 


possible between location A and B. The white line is 
the hypotenuse of a right triangle and is the most direct 
and shortest route between the two locations. In this 
case, however, the Euclidean distance between location 
A and B is actually of little value because it would be 
necessary to walk through buildings, climb over fences, 
and walk through numerous stands of vegetation to get 
from one location to another, even though it is the 
shortest distance. 


A more suitable approach to obtaining an accurate 
measurement of the functional or practical distance 
between location A and B is to a) identify a route that 
takes into account existing roads and trails and subdi- 
vide it into 7 (e.g., 6) logical line segments, and then b) 
compute the distance of these 1 segments using either 
Pythagorean theorem or Manhattan distance logic. 
The more practical distance between location A and 
location B would be one of the colored lines drawn in 
Figure 5-18. 


Numerous digital image processing programs allow the 
analyst to specify whether distances are to be comput- 
ed in multispectral feature space using a) the Pythago- 
rean theorem, or b) the Manhattan distance. For 
example, the minimum distance to mean classification 


discussed in Chapter 8 can be based on either type of 
distance measurement. 


Perimeter, Area, and Shape 
Measurement 


Perimeter Measurement 


The perimeter of a polygon is measured by determining 
the length of each of the n line segments associated 
with a polygon and then summing them: 


n 
Perimeter = » length, 
i=1 

The length of each line segment is usually computed 
using the Pythagorean theorem as previously dis- 
cussed. For example, consider the six-sided polygon 
with six vertices circumscribing a subdivision in Span- 
ish Fork, UT, shown in Figure 5-19. The coordinates of 
the six polygon vertices are listed in Table 5-7. To de- 
termine the perimeter of this polygon, the Pythagorean 
theorem was used to compute the distance associated 


(5.21) 
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TABLE 5-7 UTM coordinates of six vertices surrounding a subdivision in Spanish Fork, UT, (Figure 5-19) used to demonstrate 


how to calculate the perimeter of the complex polygon. 


UTM UTM 
X-coordinate ¥-coordinate Distance from Vertex 
Vertex Easting Northing to Vertex (e.g., 1 to 2) 
41 447487 4438722 
2 447838 4438720 351.01 
3 447833 4438541 179.07 
4 447704 4438587 136.96 
5 447687 4438538 51.87 
6 447489 4438614 212.08 
1 447487 4438722 108.02 
n 
Perimeter = > length, = 1,039 m 
i=1 


with each line segment. The perimeter of the polygon is 
1,039 m. 


Area Measurement 


The area of a rectangle on a remotely sensed image is 
computed simply by multiplying the values of its 
length and width, ie., 4 =/ x w. Another elementagy 
area computation is that of a circle, which is A = ar’. 
Complications can arise, however, when the shape of 
the polygon varies from a rectangle or circle. In the re- 
mote sensing literature, polygons are also often re- 
ferred to as areas of interest (AOIs). 


To calculate the area of a polygon (or AOJT) in remotely 
sensed imagery, the analyst typically uses a rubber- 
band tool to identify 7 vertices at unique map (x,y) or 
image (row and column) coordinates. The “contribu- 
tion” of each point (vertex) in the polygon to the area 
is computed by evaluating the x-coordinate of a vertex 
prior to the vertex under examination (x;_,) with the x- 
coordinate of the next vertex in the sequence (x;+1) and 
multiplying the result by the y-coordinate (y;) of the 
vertex under examination according to the formula: 


n 
Area = 0.5 > ii41-%—-1) ; 
i=1 


(5.22) 


It is important to point out that the very first vertex in 
the polygon uses values from the second and the mth 
(last) vertex. The last vertex uses values from the n—1 
vertex and the first vertex. 


For example, consider the use of Equation 5.22 to 
compute the area of the complex polygon surrounding 
the residential subdivision in Spanish Fork, UT (Fig- 
ure 5-19). Table 5-8 presents the X- and Y-coordinates 
of the six vertices and the contribution calculated for 
each vertex. The area of the polygon is 52,216 m* or 
approximately 5.2 hectares. This example only has six 
vertices. As you might imagine, it is more computation- 
ally intensive to calculate the area of an extremely 
complex polygon that has hundreds or even thousands 
of vertices. Also, some polygons may have polygons in- 
side them. In such cases, the area of the enclosed poly- 
gon needs to be calculated and then subtracted from 
the area of the surrounding polygon. 


After calculating the basic perimeter and area proper- 
ties of various polygons (patches) throughout a land- 
scape, it is possible to use these properties to compute 
numerous landscape ecology metrics often used in geo- 
graphic research (e.g., Frohn and Hao, 2006). 


Polygon (AOJ) area estimates extracted from rectified 
remote sensor data are more accurate than area esti- 
mates extracted from unrectified imagery. Unrectified 
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TABLE 5-8 UTM coordinates of the six vertices surrounding a subdivision in Spanish Fork, UT, (Figure 5-19) used to demon- 


strate how to calculate the area of a complex polygon. 


UTM UTM 

X-coordinate ¥-coordinate Contribution 

Vertex Easting Northing 

6 447489 4438614 
! 447487 4438722 meee aa78 
2 447838 4438720 1535797120 
3 447833 4438541 -594764494 
4 447704 4438587 -648033702 
5 447687 4438538 -954285670 
6 447489 4438614 -887722800 

1 447487 4438722 

n 
Area = 0.5 YD iO4174)-1) = 52,216 m2 
i=1 


imagery contains geometric error that can severely af- 
fect the accuracy of the areal estimates. 


Shape Measurement 


Polygonal areas of interest all have two-dimensional 
shapes with constant relationships of position and dis- 
tance of the vertices on their perimeters. One way to 
measure the shape of an AOI is to compute an index 
that relates the real-world shape (e.g., the shape of the 
six-sided polygon shown in Figure 5-19) to some regu- 
lar geometric figure such as a circle or hexagon. The 
most commonly used shape index is compactness, 
which is based on deviations from the most compact 
shape, a circle. The compactness shape index assumes a 
value of 1.0 for a perfect circle (1.e., a circle has maxi- 
mum compactness) and a value <1.0 for any less com- 
pact shape. At the opposite end of the range of the 
shape index is the straight line, which has no area anda 
shape index of zero (Earickson and Harlin, 1994). 


The shape (S) of an area of interest may be computed 
using the equation 


s- 2N(A +1) 


7 , (5.23) 


where / is the length of the longest diagonal of the 
shape and A is the area. In this example, the longest di- 
agonal within the polygon is from vertex #1 to vertex 
#3 which equals 390.48 m. The area of the polygon is 
52,216 m?. 


The shape of the six-sided polygon when compared to 
a perfect circle is 


_ 2A(52216 + 3.1416) 


. 390.48 


The shape index value of the residential subdivision is 
moderately compact. 
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6 ELECTROMAGNETIC RADIATION PRINCIPLES 
AND RADIOMETRIC CORRECTION 


The perfect remote sensing system has yet to be devel- 
oped. Also, the Earth’s atmosphere, land, and water 
are very complex and do not lend themselves well to 
being recorded by remote sensing instruments that 
have constraints such as spatial, spectral, temporal, 
and radiometric resolution. Consequently, error and 
noise can be introduced into the data acquisition pro- 
cess that can degrade the quality of the remote sensor 
data collected (Congalton and Green, 2009). This in 
turn may have an impact on the accuracy of subse- 
quent human- or machine-assisted image analysis 
(Song et al., 2001). 


The two most common types of error encountered in 
remotely sensed data are radiometric and geometric er- 
ror. Radiometric correction is concerned with improv- 
ing the accuracy of surface spectral reflectance, 
emittance, or back-scattered measurements obtained 
using a remote sensing system (Johannsen and Daugh- 
try, 2009; San and Suzen, 2010). Geometric correction 
is concerned with placing the reflected, emitted, or 
back-scattered measurements or derivative products in 
their proper planimetric (map) location. The data can 
then be associated with other geospatial information in 
a geographic information system (GIS) or in a spatial 
decision support system (SDSS) (Jensen and Jensen, 
2013). 


Radiometric and geometric correction of remotely 
sensed data are normally referred to as preprocessing 
operations because they are performed prior to infor- 
mation extraction. Image preprocessing ideally pro- 
duces a corrected image that is as close as possible, 
both radiometrically and geometrically, to the true ra- 
diant energy and spatial characteristics of the study 
area at the time of data collection. Internal and exter- 


Source: NASA 


nal errors must be identified to correct the remotely 
sensed data: 


¢ Internal errors are introduced by the remote sensing 
system itself. They are generally systematic (predict- 
able) and may be identified and then corrected 
based on prelaunch or in-flight calibration measure- 
ments. For example, -line striping in the imagery 
may be caused by a single detector that has become 
uncalibrated. In many instances, radiometric correc- 
tion can adjust for detector miscalibration. 


¢ External errors are usually introduced by phenom- 
ena that vary in nature through space and time. The 
most important external variables that can cause 
remote sensor data to exhibit radiometric and geo- 
metric error are the atmosphere, terrain elevation, 
slope, and aspect. Some external errors may be cor- 
rected by relating empirical observations made on 
the ground (i.e., radiometric and geometric ground 
control points) to sensor system measurements. 


Overview 


This chapter focuses on radiometric correction of re- 
motely sensed data. Radiometric correction requires 
knowledge about electromagnetic radiation principles 
and what interactions take place during the remote 
sensing data collection process. To be exact, it also in- 
volves knowledge about the terrain slope and aspect 
and bidirectional reflectance characteristics of the 
scene. Therefore, this chapter first reviews fundamental 
electromagnetic radiation principles. It then discusses 
how these principles and relationships are used to cor- 
rect for radiometric distortion in remotely sensed data 
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FIGURE 6-1 Energy may be transferred three ways: conduction, convection, and radiation. a) Energy may be conducted di- 
rectly from one object to another as when a pan is in direct physical contact with a hot burner. b) The Sun bathes the Earth's 
surface with radiant energy causing the air near the ground to increase in temperature. The less dense air rises, creating con- 
vectional currents in the atmosphere. c) Electromagnetic energy in the form of electromagnetic waves may be transmitted 


through the vacuum of space from the Sun to the Earth. 


caused primarily by the atmosphere and elevation. 
Chapter 7 discusses the geometric correction of remote 
sensor data. 


Electromagnetic Energy 
| teractions 


Energy recorded by remote sensing systems undergoes 
fundamental interactions that should be understood to 
properly preprocess and interpret remotely sensed da- 
ta. For example, if the energy being remotely sensed 
comes from the Sun, the energy: 


* is radiated by atomic particles at the source (the 
Sun), 

¢ travels through the vacuum of space at the speed of 
light, 

¢ interacts with the Earth’s atmosphere, 

* interacts with the Earth’s surface, 


¢ interacts with the Earth’s atmosphere once again, 
and 

¢ finally reaches the remote sensing system, where it 
interacts with various optics, filters, film emulsions, 
or detectors. 


Each of these interactions has an impact on the radio- 
metric quality of the information stored in the remote 


sensor data. It is instructive to examine each of these 
interactions that electromagnetic energy undergoes as 
it progresses from its source to the remote sensing sys- 
tem detector. 


Go nduction, G nvection, 
and Radiation 


Energy is the ability to do work. In the process of do- 
ing work, energy may be transferred from one body to 
another or from one place to another. The three basic 
ways in which energy can be transferred are conduc- 
tion, convection, and radiation (Figure 6-1). Most peo- 
ple are familiar with conduction, which occurs when 
one body (molecule or atom) transfers its kinetic ener- 
gy to another by colliding with it. This is how a metal 
pan is heated by a hot burner on a stove. During con- 
vection, the kinetic energy of bodies is transferred from 
one place to another by physically moving the bodies. 
A good example is the heating of the air near the 
ground in the morning hours. The warmer air near the 
surface rises, setting up convectional currents in the at- 
mosphere, which may produce cumulus clouds. The 
transfer of energy by electromagnetic radiation is of 
primary interest to remote sensing science because it is 
the only form of energy transfer that can take place in 
a vacuum such as the region between the Sun and the 
Earth. 
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FI GURE 62 An electromagnetic wave is complex and composed of both electric and magnetic vectors that are orthogonal 
(at 90M angles) to one another. The waves travel from the source at the speed of light (3 & 10° m/s). 


Electromagnetic Radiation 
Nb dels 


To understand how electromagnetic radiation is creat- 
ed, how it propagates through space, and how it inter- 
acts with other matter, it is useful to describe the 
processes using two different models: the wave model 
and the particle model (Vandergriff, 2014). 


Wave Model of Electromagnetic 
Energy 


In the 1860s, James Clerk Maxwell (1831-1879) con- 
ceptualized electromagnetic radiation (EMR) as an 
electromagnetic wave that travels through space at the 
speed of light, c, which is 2.99 X 10° meters per second 
(hereafter referred to as m/s) or 186,282.03 miles per 
second. A useful relation for quick calculations is that 
light travels about 1 ft per nanosecond (10° s). Light 
travels in a vacuum at a constant speed, and this speed 
is considered a universal constant. It is important to 
note that speed changes for light traveling through 
non-vacuum media such as air (0.03% slower) or glass 
(30.0% slower) (Vandergriff, 2014). The electromagnet- 
ic wave consists of two fluctuating fields—one electric 
and the other magnetic (Figure 6-2). The two vectors 
are at right angles (i.e., orthogonal) to one another, 
and both vectors are perpendicular to the direction of 
travel. 


How is an electromagnetic wave created? Electromag- 
netic radiation is generated whenever an electrical 
charge is accelerated. The wavelength (A ) of the elec- 
tromagnetic radiation depends upon the length of time 
that the charged particle is accelerated. Its frequency 
(v) depends on the number of accelerations per sec- 
ond. Wavelength is formally defined as the mean dis- 
tance between maximums (or minimums) of a roughly 
periodic pattern (Figures 6-2 and 6-3) and is normally 
measured in micrometers (Xm) or nanometers (nm). 
Frequency is the number of wavelengths that pass a 
point per unit time. A wave that sends one crest by ev- 
ery second (completing one cycle) is said to have a fre- 
quency of one cycle per second, or one hertz, 
abbreviated 1 Hz. Frequently used measures of wave- 
length and frequency are found in Table 6-1. 


The relationship between the wavelength (A ) and fre- 
quency (v) of electromagnetic radiation is based on the 
following formula, where c is the speed of light: 


c= hy, (6.1) 
v= - (6.2) 
and 
_¢ 
aaa (6.3) 


Note that frequency is inversely proportional to wave- 
length. This relationship is shown diagrammatically in 
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FIGURE 6-3 This cross-section of several electromagnetic waves illustrates the inverse relationship between wavelength (A) 
and frequency (v ). The longer the wavelength, the lower the frequency; the shorter the wavelength, the higher the frequen- 
cy. The amplitude of an electromagnetic wave is the height of the wave crest above the undisturbed position. Successive 
wave crests are numbered 1, 2, 3, and 4. An observer at the position of the clock records the number of crests that pass by in 
a second. This frequency is measured in cycles per second or hertz. 


Figure 6-3, where the longer the wavelength, the lower 
the frequency; the shorter the wavelength, the higher 
the frequency (Seeber, 2014). When electromagnetic ra- 
diation passes from one substance to another, the 
speed of light and wavelength change while the fre- 
quency remains the same. 


All objects above absolute zero (-273KC or 0 K) emit 
electromagnetic energy, including water, soil, rock, veg- 
etation, and the surface of the Sun. The Sun represents 
the initial source of most of the electromagnetic energy 
recorded by remote sensing systems (except RADAR, 
LiDAR, and SONAR) (Figures 6-4 and 6-5). We may 
think of the Sun as a 5,770 — 6,000 K blackbody (a the- 
oretical construct that absorbs and radiates energy at 
the maximum possible rate per unit area at each wave- 
length (1) for a given temperature). The total emitted 
radiation from a blackbody (M, ) measured in watts 
per m7 is proportional to the fourth power of its abso- 
lute temperature (7) measured in kelvin (K). This is 
known as the Stefan-Boltzmann law and is expressed 
as: 


M,= oT (6.4) 


where o is the Stefan-Boltzmann constant which 
equals 5.6697 X 108W m?K~. The important thing to 
remember is that the amount of energy emitted by an 
object such as the Sun or the Earth is a function of its 
temperature. The greater the temperature, the greater 
the amount of radiant energy exiting the object. The 
actual amount of energy emitted by an object is com- 
puted by summing (integrating) the area under its 
curve (Figure 6-6). It is clear from this illustration that 
the total emitted radiation from the 6,000 K Sun is far 
greater than that emitted by the 300 K Earth. 


In addition to computing the total amount of energy 
exiting a theoretical blackbody such as the Sun, we can 


determine its dominant wavelength (A,.,,.) based on 
Wien’s displacement law: 
_k 
max  -P (6.5) 


where k is a constant equaling 2,898 Xm K, and J 'is the 
absolute temperature in kelvin. Therefore, as the Sun 
approximates a 6,000 K blackbody, its dominant wave- 
length (A...) is 0.48 bh: 


max 
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TABLE 6-1 Wavelength and frequency units of measurement. 
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Wavelength (A ) 


Frequency (cycles per second) 


kilometer (km) 1,000 m 


hertz (Hz) 1 


meter (m) 1.0m 


kilohertz (kHz) 1,000 = 103 


centimeter (cm) 0.01 m=10%m 


megahertz (MHz) 1,000,000 = 10° 


millimeter (mm) 0.001 m = 103m 


gigahertz (GHz) 1,000,000,000 = 10? 


micrometer (Xm) 0.000001 = 10m 


nanometer (nm) 0.000000001 = 10°? m 


angstrom (A) 0.0000000001 = 10°'°m 
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FIGURE 6-4 The thermonuclear fusion taking place on the surface of the Sun yields a continuous spectrum of electromagnet- 
ic energy. The 5,770 — 6,000 kelvin (K) temperature of this process produces a large amount of relatively short wavelength en- 
ergy that travels through the vacuum of space at the speed of light. Some of this energy is intercepted by the Earth, where it 
interacts with the atmosphere and surface materials. The Earth reflects some of the energy directly back out to space or it 

may absorb the short wavelength energy and then re-emit it at a longer wavelength (images courtesy of NASA; line art based 


on Strahler and Strahler, 1989). 


_ 2,898 um K 


0.483 um 6,000 K 


Electromagnetic energy from the Sun travels 8 minutes 
across the intervening 93 million miles (150 million 
km) of space to reach the Earth. As shown diagram- 
matically in Figure 6-6, the Earth approximates a 300 
K (27KC) blackbody and has a dominant wavelength at 
approximately 9.66 Xm: 


_ 2,898 um K 


9.66 um 300 K 


Although the Sun has a dominant wavelength at 0.48 
Xm, it produces a continuous spectrum of electromag- 
netic radiation ranging from very short, extremely 
high-frequency gamma and cosmic waves to long, 
very-low-frequency radio waves (Figures 6-7 and 6-8). 
The Earth intercepts only a very small portion of the 
electromagnetic energy produced by the Sun. 
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Visible Portion of the Electromagnetic Spectrum from 400 to 700 nm 
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FIGURE 6-5 The portion of the electromagnetic spectrum visible by human beings is generally considered to be from 400 to 
700 nm consisting of blue (400 — 500 nm), green (500 — 600 nm), and red (600 — 700 nm) light (Ready, 2014). The image of the 
solar spectrum was observed with the Fourier Transform Spectrometer at the McMath-Pierce Solar Facility at Kitt Peak Na- 
tional Observatory, near Tucson, AZ. Each of the 50 slices covers 60 angstroms and wavelength increases from top to bottom 
along each slice. Note that it contains relatively narrow, dark Fraunhofer lines named for the German physicist Joseph von 
Fraunhofer (1787-1826) who “mapped” the location of over 570 of them in the solar spectrum. Later scientists discovered 
that the dark lines are caused by chemical element absorption in the upper layers of the Sun and some absorption by oxygen 
molecules in the Earth’s atmosphere (N. A. Sharp, NOAO/NSO/Kitt Peak FTS/AURA/NSF). 


As mentioned in Chapter 1, in remote sensing research 
we often specify a particular region of the electromag- 
netic spectrum (e.g., red light) by identifying a begin- 
ning and ending wavelength (or frequency) and then 
attaching a description. This wavelength (or frequency) 
interval in the electromagnetic spectrum is commonly 
referred to as a band, channel, or region. The major 


subdivisions of visible light are presented diagrammati- 
cally in Figures 6-5, 6-6, and 6-7 and summarized in 
Table 6-2. For example, we generally think of visible 
light as being composed of energy in the blue (0.4 — 0.5 
kim), green (0.5 — 0.6 Xm), and red (0.6 — 0.7 Xm) bands 
of the electromagnetic spectrum. Similarly, reflected 
near-infrared energy in the region from 0.7 to 1.3 Km is 
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TABLE 6-2 Methods of describing the color spectrum (after Nassau, 1983; 1984). 


Wavelength Descriptions 


Energy Descriptions 


Blue 


22,200 


Angstrom Nanometer Micrometer Frequency Wave Electron 
Col (A) (nm) (Xm) Hz (X 1,014) Number Volt 
olor (y cm”) (eV) 
Ultraviolet, sw Zoo 254 0.254 11.82 39,400 4.89 
Ultraviolet, lw 3,660 366 0.366 8.19 27,300 3.39 


commonly used to expose black-and-white and color- 
infrared-sensitive film. 


The middle-infrared region (often referred to as the 
short wavelength infrared, SWIR) includes energy with 
a wavelength of 1.3 to 3 Xm. The thermal infrared re- 
gion has two very useful bands at 3 to 5m and 8 to 14 
im. The microwave portion of the spectrum consists of 
much longer wavelengths (1 mm—1I m). The radio-wave 
portion of the spectrum may be subdivided into UHF, 
VHF, radio (HF), LF, and ULF frequencies. 


The spectral resolution of most remote sensing systems 
is described in terms of bands of the electromagnetic 
spectrum. The band specifications for several of the 
most important remote sensing systems were summa- 
rized in Chapter 2. 


Electromagnetic energy may be described not only in 
terms of wavelength and frequency but also in photon 
energy units such as joules (J) and electron volts (eV), 
as shown in Figure 6-8. Important mass, energy, and 
power conversions are summarized in Table 6-3. 


The Particle Model: Radiation from 
Atomic Structures 


In Opticks (1704), Sir Isaac Newton stated that light 
was a stream of particles, or corpuscles, traveling in 
straight lines. He also knew that light had wavelike 
characteristics based on his work with glass plates. 
Nevertheless, during the 200 years before 1905, light 
was thought of primarily as a smooth and continuous 
wave. Albert Einstein (1879-1955) found that when 
light interacts with electrons, it has a different charac- 
ter. He concluded that when light interacts with matter, 
it behaves as though it is composed of many individual 
bodies called photons, which carry particle-like proper- 
ties such as energy and momentum. As a result, most 
physicists today would answer the question, “What is 
light?” by saying that light is a particular kind of mat- 
ter. Thus, we sometimes describe electromagnetic ener- 
gy in terms of its wavelike properties. But when the 
energy interacts with matter, it is useful to describe it as 
discrete packets of energy, or quanta. It is practical to 
review how electromagnetic energy is generated at the 
atomic level; this provides insight as to how light inter- 
acts with matter. 
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FIGURE 6-6 Blackbody radiation curves for several objects, 
including the Sun and the Earth, which approximate 6,000 K 
and 300 K blackbodies, respectively. The area under each 
curve may be summed to compute the total radiant energy 
(M, ) exiting each object (Equation 6.4). Thus, the Sun pro- 
duces more radiant exitance than the Earth because its tem- 
perature is greater. As the temperature of an object 
increases, its dominant wavelength (A shifts toward the 
shorter wavelengths of the spectrum. 


max ) 


Electrons are the tiny negatively charged particles that 
move around the positively charged nucleus of an atom 
(Figure 6-9). Atoms of different substances are made 
up of varying numbers of electrons arranged in differ- 
ent ways. The interaction between the positively 
charged nucleus and the negatively charged electron 
keeps the electron in orbit. While its orbit is not explic- 
itly fixed, each electron’s motion is restricted to a defi- 
nite range from the nucleus. The allowable orbital 
paths of electrons moving around an atom might be 
thought of as energy classes or levels (Figure 6-9a). In 
order for an electron to climb to a higher class, work 
must be performed. However, unless an amount of en- 
ergy is available to move the electron up at least one en- 
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FIGURE 6-7 The Sun approximates a 6,000 K blackbody 
with a dominant wavelength of about 0.48 Xm. The Earth 
approximates a 300 K blackbody with a dominant wave- 
length of about 9.66 im. The 6,000 K Sun produces approx- 
imately 41% of its energy in the visible region from 0.4 to 
0.7 Km (blue, green, and red light). The other 59% of the en- 
ergy is in wavelengths shorter than blue light (<0.4 Mm) and 
longer than red light (>0.7 Xm). Our eyes are only sensitive 
to light from 0.4 to 0.7 Xm (after Strahler and Strahler, 1989). 
Fortunately, it is possible to make remote sensor detectors 
sensitive to energy in these nonvisible regions of the spec- 
trum. 


ergy level, it will accept no work. If a sufficient amount 
of energy is received, the electron will jump to a new 
level and the atom is said to be excited (Figure 6-9b). 

Once an electron is in a higher, orbit, it possesses po- 
tential energy. After about 10°° seconds, the electron 
falls back to the atom’s lowest empty energy level or 
orbit and gives off radiation (Figure 6-9c). The wave- 
length of radiation given off is a function of the 
amount of work done on the atom, 1.e., the quantum of 
energy it absorbed to cause the electron to become ex- 
cited and move to a higher orbit. 


Electron orbits are like the rungs of a ladder. Adding 
energy moves the electron up the energy ladder; emit- 
ting energy moves it down. However, the energy ladder 
differs from an ordinary ladder in that its rungs are un- 
evenly spaced. This means that the energy an electron 
needs to absorb, or to give up, in order to jump from 
one orbit to the next may not be the same as the energy 
change needed for some other step. Furthermore, an 
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FIGURE 6-8 The electromagnetic spectrum and the photon energy of visible light. The Sun produces a continuous spectrum 
of energy from gamma rays to radio waves that continually bathe the Earth in energy. The visible portion of the spectrum may 
be measured using wavelength (measured in micrometers or nanometers, i.e., Xm or nm) or electron volts (eV). All of the units 


are interchangeable. 


electron does not necessarily use consecutive rungs. In- 
stead, it follows what physicists call selection rules. In 
many cases, an electron uses one sequence of rungs as 
it climbs the ladder and another sequence as it de- 
scends (Nassau, 1983). The energy that is left over 
when the electrically charged electron moves from an 
excited state (Figure 6-9b) to a de-excited state (Figure 
6-9c) is emitted by the atom as a packet of electromag- 
netic radiation, a particle-like unit of light called a pho- 
ton. Every time an electron jumps from a higher to a 
lower energy level, a photon moves away at the speed 
of light. 


Somehow an electron must disappear from its original 
orbit and reappear in its destination orbit without ever 
having to traverse any of the positions in between. This 
process is called a quantum leap or quantum jump. If the 
electron leaps from its highest excited state to the 


ground state in a single leap, it will emit a single pho- 
ton of energy. It is also possible for the electron to leap 
from an excited orbit to the ground state in a series of 
jumps (e.g., from 4 to 2 to 1). If it takes two leaps to get 
to the ground state, then each of these jumps will emit 
photons of somewhat less energy. The energies emitted 
in the two jumps must sum to the total of the single 
large jump. 


Niels Bohr (1885-1962) and Max Planck recognized 
the discrete nature of exchanges of radiant energy and 
proposed the quantum theory of electromagnetic radia- 
tion. This theory states that energy is transferred in dis- 
crete packets called quanta or photons, as discussed. 
The relationship between the frequency of radiation 
expressed by wave theory and the quantum is: 


O=hy (6.6) 
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FIGURE 6-9 a-c) A photon of electromagnetic energy is emitted when an electron in an atom or molecule drops from a 
higher-energy state to a lower-energy state. The light emitted (i.e., its wavelength) is a function of the changes in the energy 
levels of the outer, valence electron. For example, yellow light is produced from a sodium vapor lamp in Figure 6-10. d) Mat- 
ter can also be subjected to such high temperatures that electrons, which normally move in captured, nonradiating orbits, are 
broken free. When this happens, the atom remains with a positive charge equal to the negatively charged electron that es- 
caped. The electron becomes a free electron, and the atom is called an ion. If another free electron fills the vacant energy lev- 
el created by the free electron, then radiation from all wavelengths is produced, i.e., a continuous spectrum of energy. The 
intense heat at the surface of the Sun produces a continuous spectrum in this manner. 


where Q is the energy of a quantum measured in joules, 
his the Planck constant (6.626 X 107? J s), and v is the 
frequency of the radiation. Referring to Equation 6.3, 
we can multiply the equation by A/h, or 1, without 
changing its value: 


_ he 

hy (6.7) 
By substituting QO for hv (from Equation 6.6), we can 
express the wavelength associated with a quantum of 
energy as: 


he 
== 6.8 
O (6.8) 
or 
_ he 
= te (6.9) 


Thus, we see that the energy of a quantum is inversely 
proportional to its wavelength, i.e., the longer the 
wavelength, the lower its energy content. This inverse 
relationship is important to remote sensing because it 
suggests that it is more difficult to detect the longer- 
wavelength energy being emitted at thermal infrared 
wavelengths than those at shorter visible wavelengths. 
In fact, it might be necessary to have the sensor look at 
or dwell longer on the parcel of ground if we are trying 
to measure the longer wavelength energy. 


Substances have color because of differences in their 
energy levels and the selection rules. For example, con- 
sider energized sodium vapor that produces a bright 
yellow light that is used in some street lamps. When a 
sodium vapor lamp is turned on, several thousand 
volts of electricity energize the vapor. The outermost 
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TABLE 6-3 Mass, energy, and power conversions. 


Conversion from English to SI Units 


To get: Multiply: By: 
newtons? pounds 4.448 
joules BTUs¢ 1055 
joules calories® 4.184 
joules kilowatt-hours® 3.6m 10° 
joules foot-pounds* 1.356 
joules horsepower9 745.7 


Conversion from SI to English Units 


To get: Multiply: By: 

BTUs joules 0.00095 
calories joules 0.2390 
kilowatt-hours joules 2.78 8 107 
foot-pounds joules 0.7375 
horsepower watts 0.00134 


“newton: force needed to accelerate a mass of | kg by 1 
ms? 

boule: a force of 1 newton acting through | meter. 

“British thermal unit, or BTU: energy required to raise 
the temperature of | pound of water by | degree Fahr- 
enheit. 

dcalorie: energy required to raise the temperature of | 
kilogram of water by | degree Celsius. 

°kilowatt-hour: 1,000 joules per second for | hour. 

foot-pound: a force of 1 pound acting through | foot. 

Shorsepower: 550 foot-pounds per second. 


electron in each energized atom of sodium vapor 
climbs to a higher rung on the energy ladder and then 
returns down the ladder in a certain sequence of rungs, 
the last two of which are 2.1 eV apart (Figure 6-10). 
The energy released in this last leap appears as a pho- 
ton of yellow light with a wavelength of 0.58 Xm with 
2.1 eV of energy (Nassau, 1983). 


Matter can be heated to such high temperatures that 
electrons that normally move in captured, nonradiat- 
ing orbits break free. This is called the photoelectric ef- 
fect (Figure 6-9d). When this happens, the atom 
remains with a positive charge equal to the negatively 
charged electron that escaped. The electron becomes a 


Creation of Light from Atomic Particles 
in a Sodium Vapor Lamp 
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FIGURE 6-10 Creation of light from atomic particles in a 
sodium vapor lamp. After being energized by several thou- 
sand volts of electricity, the outermost electron in each en- 
ergized atom of sodium vapor climbs to a high rung on the 
energy ladder and then returns down the ladder in a pre- 
dt able fashio The las two rungs in the descent are 2.1 
eV apart. This produces a photon of yellow light, which has 
2.1 eV of energy (see Table 6-2). 


free electron and the atom is called an ion. In the ultra- 
violet and visible (blue, green, and red) parts of the 
electromagnetic spectrum, radiation is produced by 
changes in the energy levels of the outer valence elec- 
trons. The wavelengths of energy produced are a func- 
tion of the particular orbital levels of the electrons 
involved in the excitation process. If the atoms absorb 
enough energy to become ionized and if a free electron 
drops in to fill the vacant energy level, then the radia- 
tion given off is unquantized and a continuous spec- 
trum is produced rather than a band or a series of 
bands. Every encounter of one of the free electrons 
with a positively charged nucleus causes rapidly chang- 
ing electric and magnetic fields, so that radiation at all 
wavelengths is produced. The hot surface of the Sun is 
largely a plasma in which radiation of all wavelengths is 
produced. As previously shown in Figure 6-8, the spec- 
trum of a plasma like the Sun is continuous. 


In atoms and molecules, the changes in electron orbits 
produce the shortest wavelength radiation, molecule 
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vibrational motion changes produce near-infrared and/ 
or middle-infrared energy, and rotational motion 
changes produce long-wavelength infrared or micro- 
wave radiation. 


Atmospheric Energy—Matter 
Interactions 


Radiant energy is the capacity of radiation within a 
spectral band to do work. Once electromagnetic radia- 
tion is generated, it is propagated through the Earth’s 
atmosphere almost at the speed of light in a vacuum. 
Unlike a vacuum in which nothing happens, however, 
the atmosphere may affect not only the speed of radia- 
tion but also its wavelength, its intensity, and its spec- 
tral distribution. The electromagnetic radiation may 
also be diverted from its original direction due to re- 
fraction. 


Refraction 


The speed of light in a vacuum is 3 & 10° m/s. When 
electromagnetic radiation encounters substances of 
different density, like air and water, refraction may take 
place. Refraction refers to the bending of light when it 
passes from one medium to another of different densi- 
ty. Refraction occurs because the media are of differing 
densities and the speed of EMR is different in each. 
The index of refraction (n) is a measure of the optical 
density of a substance. This index is the ratio of the 
speed of light in a vacuum, c, to the speed of light in a 
substance such as the atmosphere or water, c,,: 


n=<., (6.10) 


The speed of light in a substance can never reach the 
speed of light in a vacuum. Therefore, its index of re- 
fraction must always be greater than 1. For example, 
the index of refraction for the atmosphere is 1.0002926 
and 1.33 for water. Light travels more slowly through 
water because of water’s higher density. 


Refraction can be described by Snell’s law, which states 
that for a given frequency of light (we must use fre- 
quency since, unlike wavelength, it does not change 
when the speed of light changes), the product of the in- 
dex of refraction and the sine of the angle between the 
ray and a line normal to the interface is constant: 

(6.11) 


n,sin 0, = ny sin 0,. 


From Figure 6-11 we can see that a nonturbulent at- 
mosphere can be thought of as a series of layers of gas- 
es, each with a slightly different density. Any time 


Atmospheric Refraction 


Normal to 
the surface 


Incident 
radiant energy 


n, = index of 
refraction for 
this layer of 
the atmosphere 


Optically 
less dense 
atmosphere 


n, 
Optically 
more dense 
atmosphere 


n 
Optically 
less dense 

atmosphere 


Path of energy affected 
by atmospheric refraction 


FIGURE 6-11 Refraction in three nonturbulent atmospher- 
ic layers. The incident energy is bent from its normal trajec- 
tory as it travels from one atmospheric layer to another. 
Snell's law can be used to predict how much bending will 
take place, based on a knowledge of the angle of incidence 
(8) and the index of refraction of each atmospheric layer, 
N41, Ng, N3. 


energy is propagated through the atmosphere for any 
appreciable distance at any angle other than vertical, 
refraction occurs. 


The amount of refraction is a function of the angle 
made with the vertical (8), the distance involved (in 
the atmosphere the greater the distance, the more 
changes in density), and the density of the air involved 
(air is usually more dense near sea level). Serious errors 
in location due to refraction can occur in images 
formed from energy detected at high altitudes or acute 
angles. However, these location errors are predictable 
by Snell’s law and thus can be removed. Notice that 


n,sin 9, 
=, (6.12) 


ig 


Therefore, if an analyst knows the index of refraction 
of medium 7, and np and the angle of incidence of the 
energy to medium nj, it is possible to predict the 
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Atmospheric Layers and Constituents 


O3, stratospheric aerosols 


Altitude Above Sea Level 


Terrain Surface 


FIGURE 6-12 Major subdivisions of the atmosphere and 
the types of molecules and aerosols found in each layer 
(based on Miller and Vermote, 2002). 


amount of refraction that will take place (sin 0, ) in 
medium 7 using trigonometric relationships. Interest- 
ingly, most image analysts never concern themselves 
with computing the index of refraction. 


Scattering 


One very serious effect of the atmosphere is the scatter- 
ing of radiation by atmospheric particles. Scattering 
differs from reflection in that the direction associated 
with scattering is unpredictable, whereas the direction 
of reflection (to be defined shortly) is predictable. 
There are essentially three types of scattering: Ray- 
leigh, Mie, and nonselective scattering. Major subdivi- 
sions of the atmosphere and the types of molecules and 
aerosols found in each layer are shown in Figure 6-12 
(Miller and Vermote, 2002). The relative size of the 
wavelength of the incident electromagnetic radiation, 
the diameter of the gases, water vapor, and/or dust 
with which the energy interacts, and the type of scatter- 
ing that should occur are summarized in Figure 6-13. 


Rayleigh scattering (often referred to as molecular scat- 
tering) occurs when the effective diameter of the matter 
(usually air molecules such as oxygen and nitrogen in 
the atmosphere) are many times smaller (usually < 0.1) 
than the wavelength of the incident electromagnetic ra- 
diation (Figure 6-13a). Rayleigh scattering is named af- 
ter the English physicist Lord Rayleigh, who offered 
the first coherent explanation for it. All scattering is ac- 
complished through absorption and re-emission of ra- 
diation by atoms or molecules in the manner 
previously described in the section on radiation from 
atomic structures. It is impossible to predict the direc- 
tion in which a specific atom or molecule will emit a 
photon, hence scattering. The energy required to excite 
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FIGURE 6-13 Type of scattering is a function of 1) the 
wavelength of the incident radiant energy, and 2) the size of 
the gas molecule, dust particle, and/or water vapor droplet 
encountered. 


an atom is associated with powerful short-wavelength, 
high-frequency radiation. 


The amount of Rayleigh scattering in the atmosphere 
in optical wavelengths (0.4-0.7 Xm) may be computed 
using the Rayleigh scattering cross-section (t,, ) algo- 
rithm (Cracknell and Hayes, 1993): 


3,2 2 
= cee (6.13) 
(3NA*) 
where n = refractive index, N = number of air mole- 
cules per unit volume, and ~’ = wavelength. The 


amount of scattering is inversely related to the fourth 
power of the radiation’s wavelength. For example, ul- 
traviolet light at 0.3 Km is scattered approximately 16 
times more than red light at 0.6 Xm, i.e., (0.6/0.3)4 = 16. 
Blue light at 0.4 Xm is scattered about 5 times more 
than red light at 0.6 Xm, ie., (0.6/0.4)4 = 5.06. The 
amount of Rayleigh scattering expected throughout 
the visible part of the spectrum (0.4 — 0.7 Xm) is shown 
in Figure 6-14. 


Most Rayleigh scattering by gas molecules takes place 
in the atmosphere 2 to 8 km above the ground (Figure 
6-12). Rayleigh scattering is responsible for the blue 
appearance of the sky. The shorter violet and blue 
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FIGURE 6-14 The intensity of Rayleigh scattering varies in- 
versely with the fourth power of the wavelength (A~“). 


wavelengths are more efficiently scattered than the lon- 
ger orange and red wavelengths. When we look up on a 
cloudless day and admire the blue sky, we are witness- 
ing the preferential scattering of the short-wavelength 
sunlight. Rayleigh scattering is also responsible for red 
sunsets. Since the atmosphere is a thin shell of gravita- 
tionally bound gas surrounding the solid Earth, sun- 
light must pass through a longer slant path of air at 
sunset (or sunrise) than at noon. Since the violet and 
blue wavelengths are scattered even more during their 
now longer path through the air than when the Sun is 
overhead, what we see when we look toward the sunset 
is the residue—the wavelengths of sunlight that are 
hardly scattered away at all, especially the oranges and 
reds. 


Mie scattering (sometimes referred to as nonmolecular 
or aerosol particle scattering) takes place in the lower 
4.5 km of the atmosphere, where there may be many es- 
sentially spherical particles present with diameters ap- 
proximately equal to the size of the wavelength of the 
incident energy (Figure 6-13b). The actual size of the 
particles may range from 0.1 to 10 times the wave- 
length of the incident energy. For visible light, the main 
scattering agents are dust and other particles ranging 


from a few tenths of a micrometer to several microme- 
ters in diameter. The amount of scatter is greater than 
Rayleigh scatter, and the wavelengths scattered are lon- 
ger. The greater the amount of smoke and dust parti- 
cles in the atmospheric column, the more that violet 
and blue light will be scattered away and only the lon- 
ger wavelength yellow, orange, and red light will reach 
our eyes. Pollution also contributes to beautiful sunsets 
and sunrises. 


Nons elective scattering takes place in the lowest por- 
tions of the atmosphere where there are particles >10 
times the wavelength of the incident electromagnetic 
radiation (Figure 6-13c). This type of scattering is non- 
selective, i.e., all wavelengths of light are scattered, not 
just blue, green, or red. Thus, the water droplets and 
ice crystals that make up clouds and fog banks scatter 
all wavelengths of visible light equally well, causing the 
cloud to appear white. Nonselective scattering of ap- 
proximately equal proportions of blue, green, and red 
light always appears as white light to the casual ob- 
server. This is the reason why putting our automobile 
high beams on in fog only makes the problem worse as 
we nonselectively scatter even more light into our visu- 
al field of view. 


Scattering is a very important consideration in remote 
sensing investigations. It can severely reduce the infor- 
mation content of remotely sensed data to the point 
that the imagery loses contrast and it becomes difficult 
to differentiate one object from another. 


Absorption 


Absorption is the process by which radiant energy is ab- 
sorbed and converted into other forms of energy. The 
absorption of the incident radiant energy may take 
place in the atmosphere or on the terrain. An absorp- 
tion band is a range of wavelengths (or frequencies) in 
the electromagnetic spectrum within which radiant en- 
ergy is absorbed by a substance. The effects of water 
(HO), carbon dioxide (CO>), oxygen (O>), ozone (O3), 
and nitrous oxide (N,O) on the transmission of light 
through the atmosphere are summarized in Figure 6- 
15a. The cumulative effect of the absorption by the 
various constituents can cause the atmosphere to 
“close down” completely in certain regions of the spec- 
trum. This is bad for remote sensing because no energy 
is available to be sensed. Conversely, in the visible por- 
tion of the spectrum (0.4 — 0.7 Xm), the atmosphere 
does not absorb all of the incident energy but transmits 
it rather effectively. Portions of the spectrum that 
transmit radiant energy effectively are called atmo- 
spheric windows. 


Absorption occurs when incident energy of the same 
frequency as the resonant frequency of an atom or 
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Absorption of the Sun’s Incident Electromagnetic Energy in 
the Region from 0.1 to 30 um by Various Atmospheric Gases 
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FIGU RE 645 a) The absorption of the Sun’s incident electromagnetic energy in the region from 0.1 to 30 Km by various at- 
mospheric gases. The first four graphs depict the absorption characteristics of N9O, Oz and O3, CO, and H3O, while the fi- 
nal graphic depicts the cumulative result of all these constituents being in the atmosphere at one time. The atmosphere 
essentially “closes down” in certain portions of the spectrum while “atmospheric windows” exist in other regions that trans- 
mit incident energy effectively to the ground. It is within these windows that remote sensing systems must function. b) The 
combined effects of atmospheric absorption, scattering, and reflectance reduce the amount of solar irradiance reaching the 
Earth's surface at sea level (based on Slater, 1980). 


molecule is absorbed, producing an excited state. Ifin- tion occurs. When dealing with a medium like air, 
stead of reradiating a photon of the same wavelength, absorption and scattering are frequently combined 
the energy is transformed into heat motion and is sub- — into an extinction coefficient (Konecny, 2014). Trans- 
sequently reradiated at a longer wavelength, absorp- _ mission is inversely related to the extinction coefficient 
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Specular versus Diffuse ReXectance 
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FIGURE 6-16 The nature of specular and diffuse reflectance. 


times the thickness of the layer. Certain wavelengths of 
radiation are affected far more by absorption than by 
scattering. This is particularly true of infrared and 
wavelengths shorter than visible light. The combined 
effects of atmospheric absorption, scattering, and re- 
flectance (from cloud tops) can dramatically reduce the 
amount of solar radiation reaching the Earth’s surface 
at sea level, as shown in Figure 6-1 5b. 


Energy reaching the remote sensing system must pass 
through the atmosphere twice. Therefore, it is common 
to identify two atmospheric transmission coefficients: 
one for energy coming into the atmosphere (7, ) at an 
incidence angle related to the source of the energy (e.g., 
the Sun) and one for the atmosphere that the Earth 
surface—reflected or emitted energy must pass through 
to reach the remote sensor system (7, ). We will see 
how important these two parameters are in the discus- 
sion on atmospheric correction later in this chapter. 


Reflectance 


Reflectance is the process whereby radiation “bounces 
off” an object like the top of a cloud, a water body, or 
the terrestrial Earth. Actually, the process is more com- 
plicated, involving reradiation of photons in unison by 
atoms or molecules in a layer approximately one-half 
wavelength deep. Reflection exhibits fundamental 
characteristics that are important in remote sensing. 
First, the incident radiation, the reflected radiation, 
and a vertical to the surface from which the angles of 
incidence and reflection are measured all lie in the 
same plane. Second, the angle of incidence and the an- 
gle of reflection (exitance) are approximately equal, as 
shown in Figure 6-16. 


There are various types of reflecting surfaces. Specular 
reflection occurs when the surface from which the radi- 
ation is reflected is essentially smooth (i.e., the average 
surface-profile height is several times smaller than the 
wavelength of radiation striking the surface). Several 
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TABLE 6-4 Radiometric concepts (based on Colwell, 1983; Vandergriff, 2014). 


Name Symbol Units Concept 
Radiant energy On joules, J Capacity of radiation within a specified 
spectral band to do work. 
Radiant flux ®, watts, W Time rate of flow of energy onto, off of, or 
through a surface. 
Radiant flux density at the surface 
Irradiance Ey watts per square meter, Radiant flux incident upon a surface per 
Wm | unit area of that surface. 
Radiant exitance M. watts per square meter, Radiant flux leaving a surface per unit area 
r Wm | of that surface. 
Radiance (spectral) Ly watts per square meter, Radiant intensity per unit of projected 
per steradian, W m~ sr source area in a specified direction. 
a A ‘ 4 ® fi 

Hemispherical reflectance Py dimensionless re lected, 

a) 
Hemispherical transmittance T) dimensionless transmitted, 

Oo. 

Mh 

Hemispherical absorptance oy dimensionless ® absorbed, 
Oo. 
uN 


features, such as very calm water bodies, act like near- 
perfect specular reflectors (Figure 6-l6ab). If there are 
very few ripples on the surface, the incident energy will 
leave the water body at an angle equal and opposite to 
the incident energy (Pedrotti, 2014). We know this oc- 
curs from our personal experience. If we shine a flash- 
light at night on a tranquil pool of water, the light will 
bounce off the surface and into the trees across the way 
at an angle equal to and opposite from the incident ra- 
diation angle. 


If the surface has a large surface height relative to the 
size of the wavelength of the incident energy, the re- 
flected rays go in many directions, depending on the 
orientation of the smaller reflecting surfaces. This dif- 
f useref lation does not yield a mirror image, but in- 
stead produces diffused radiation (Figure 6-16c). 
White paper, white powders, and other materials reflect 
visible light in this diffuse manner. If the surface is so 
rough that there are no individual reflecting surfaces, 
then unpredictable scattering may occur. Lambert de- 
fined a perfectly diffuse surface; hence, the commonly 
designated Lambertian surface is one for which the ra- 
diant flux leaving the surface is constant for any angle 
of reflectance (Figure 6-16d). 


A considerable amount of incident radiant flux from 
the Sun is reflected from the tops of clouds and other 
materials in the atmosphere. A substantial amount of 


this energy is reradiated back to space. As we shall see, 
the specular and diffuse reflection principles that apply 
to clouds also apply to the terrain. 


Terrain Energy—Matter 
Interactions 


The time rate of flow of energy onto, off of, or through 
a surface is called radiant flux (® ) and is measured in 
watts (W) (Table 6-4). The characteristics of the radi- 
ant flux and what happens to it as it interacts with the 
Earth’s surface is of critical importance in remote sens- 
ing. In fact, this is the fundamental focus of much re- 
mote sensing research. By carefully monitoring the 
exact nature of the incoming (incident) radiant flux in 
selective wavelengths and how it interacts with the ter- 
rain, it is possible to learn important information 
about the terrain. 


Radiometric quantities have been identified that allow 
analysts to keep a careful record of the incident and ex- 
iting radiant flux (Table 6-4). We begin with the simple 
radiation budget equation, which states that the total 
amount of radiant flux in specific wavelengths (A ) inci- 
dent to the terrain (®, ) must be accounted for by 
evaluating the amount 6f radiant flux reflected from 


202 INTRODUCTORY DIGITAL IMAGE PROCESSING 


the surface ( ®, eftected, )» the amount of radiant flux ab- 
sorbed by the surface (®,.,.heq. )s and the amount of 


radiant flux transmitted through the surface 
( ® ransmitted, ) : 
2; = P reflected, * P absorbed, = Pansmitted, - (6.14) 


It is important to note that these radiometric quanti- 
ties are based on the amount of radiant energy incident 
to a surface from any angle in a hemisphere (i.e., a half 
of a sphere). 


Hemispherical Reflectance, 
Absorptance, and Transmittance 


Hemispherical reflectance (p,, ) is defined as the dimen- 
sionless ratio of the radiant flux reflected from a sur- 
face to the radiant flux incident to it (Table 6-4): 


_ P reflected, 
P.- —@, 


1, 


(6.15) 


Hemispherical transmittance (t,) is defined as the di- 
mensionless ratio of the radiant flux transmitted 
through a surface to the radiant flux incident to it: 


D ransmitted, 


ee (6.16) 


Hemispherical absorptance (a, ) is defined by the di- 
mensionless relationship: 


® 
O, = absorbed, (6.17) 
i 
or 


a, = 1-(7,+1,). (6.18) 


These definitions imply that radiant energy must be 
conserved, i.e., it is either returned back by reflection, 
transmitted through a material, or absorbed and trans- 
formed into some other form of energy inside the ter- 
rain. The net effect of absorption of radiation by most 
substances (e.g., plants, soil, rocks, water) is that the 
energy is converted into heat, causing a subsequent rise 
in the substance’s temperature. 


These radiometric quantities are useful for producing 
general statements about the spectral reflectance, ab- 
sorptance, and transmittance characteristics of terrain 


features. In fact, if we take the simple hemispherical re- 
flectance equation and multiply it by 100, we obtain an 
expression for percent reflectance (p ts ): 


_ P reflected, 
p Dog Q. 


1 


x 100, (6.19) 


which is often used in remote sensing research to de- 
scribe the spectral reflectance characteristics of various 
phenomena. 


Examples of spectral percent reflectance curves for se- 
lected urban-suburban phenomena such as concrete, 
sand, sandy clay soil, red brick, Centipede grass, artifi- 
cial turf, water, and asphalt shingles are shown in Fig- 
ure 6-17. Spectral reflectance curves provide no 
information about the absorption and transmittance of 
the radiant energy. But because many of the sensor sys- 
tems such as cameras and some multispectral scanners 
record only reflected energy, this information is still 
quite valuable and can form the basis for object identi- 
fication and assessment. For example, it is clear from 
Figure 6-17 that the Centipede grass reflects only ap- 
proximately 3 to 4% of the incident red radiant energy 
(0.6 — 0.7 Xm) while reflecting up to approximately 35% 
of the incident near-infrared radiant flux (0.7 — 0.9 
Mim). If we wanted to discriminate between grass and 
artificial turf, the ideal portion of the spectrum to re- 
motely sense in would be the near-infrared region be- 
cause artificial turf reflects only about 5 percent of the 
incident near-infrared energy. This would cause a 
black-and-white infrared image of the terrain to dis- 
play Centipede grass in bright tones and the artificial 
turf in darker tones. Please note how concrete reflects 
approximately 35% of the incident energy in the blue, 
green, and red parts of the spectrum, causing it to ap- 
pear gray when humans view it a natural color photo- 
graph. This will be demonstrated in a subsequent 
section in this chapter. 


Hemispherical reflectance, transmittance, and absorp- 
tance of radiometric quantities do not provide infor- 
mation about the exact amount of energy reaching a 
specific area on the ground from a specific direction or 
about the exact amount of radiant flux exiting the 
ground in a certain direction. Remote sensing systems 
can be located in space only at a single point in time, 
and they usually look only at a relatively small portion 
of the Earth at a single instant in time. Therefore, it is 
important to refine our radiometric measurement tech- 
niques so that more precise radiometric information 
can be extracted from the remotely sensed data. This 
requires the introduction of several radiometric quanti- 
ties that provide progressively more precise radiometric 
information. 
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Spectral ReXectance of Selected Materials 
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FIGURE 6-17 Spectral reflectance curves for selected urban-suburban phenomena in the region 400 — 2,500 nm. 


Radiant Flux Density 


A flat area (e.g., 1 X 1 min dimension) being bathed in 
radiant flux (® ) in specific wavelengths from the Sun 
is Shown in Figure 6-18. The amount of radiant flux in- 
tercepted divided by the area of the plane surface is the 
average radiant flux density. 


Irradiance and Exitance 


The amount of radiant flux incident upon a surface per 
unit area of that surface is called irradiance (E, ): 


QO. 
is 
i= vt (6.20) 


The amount of radiant flux leaving a surface per unit 
area of that surface is called exitance (M,): 


© 4 


M, = —. (6.21) 


Both quantities are usually measured in watts per me- 
ter squared (W m7”). Although we do not have infor- 
mation on the direction of either the incoming or 
outgoing radiant energy (i.e., the energy can come and 
go at any angle throughout the entire hemisphere), we 
have now refined the measurement to include informa- 
tion about the size of the study area of interest on the 
ground in m*. Next we need to refine our radiometric 
measurement techniques to include information on 
what direction the radiant flux is leaving (exiting) the 
study area. 


Radiance 


Radiance is the most precise remote sensing radiomet- 
ric measurement. Radiance (L, ) is the radiant intensity 
per unit of projected source area in a specified direc- 
tion. It is measured in watts per meter squared per ste- 
radian (W m7? sr’). The concept of radiance can be 
understood by examining the components shown in 
Figure 6-19. First, the radiant flux leaves the projected 
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Radiant Flux Density 
Radiant Mux, ®,, 
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FI@ RE 648 The concept of radiant flux density for an 
area on the surface of the Earth. a) Irradiance is a measure 
of the amount of radiant flux incident upon a surface per 
unit area of the surface measured in watts m’*. b) Exitance is 
a measure of the amount of radiant flux leaving a surface 
per unit area of the surface measured in watts m”. 


source area in a specific direction toward the remote 
sensor. We are not concerned with any other radiant 
flux that might be leaving the source area in any other 
direction. We are interested only in the radiant flux in 
certain wavelengths (®, ) leaving the projected source 
area (A) within a certain direction (cos 0) and solid 
angle (Q.) (Milman, 1999): 


ue) 
Q 


L, = ——.. 
4 Acos 0 


(6.22) 


One way of visualizing the solid angle is to consider 
what you would see if you were in an airplane looking 
through a telescope at the ground. Only the energy that 
exited the terrain and came up to and through the tele- 
scope in a specific solid angle (measured in steradians) 
would be intercepted by the telescope and viewed by 
your eye. Therefore, a solid angle is similar to a three- 
dimensional cone (or tube) that funnels radiant flux 
from a specific point source on the terrain toward the 
sensor system. Hopefully, energy from the atmosphere 
or other terrain features does not become scattered 


Radiance 


Normal 


to surface . 
Radiant Xux, @ nN 


Side view 
of source 
area, A 


®, Projected source area = A cos 0 


L, = 
4 Acos ) 

FIGURE 64 9 The concept of radiance leaving a specific 

projected source area on the ground, in a specific direction, 

and within a specific solid angle. 


into the solid angle field of view and contaminate the 
radiant flux from the area of interest on the ground. 
Unfortunately, this is not often the case because scat- 
tering in the atmosphere and from other nearby areas 
on the ground can contribute spurious spectral energy, 
which enters into the solid angle field of view. 


Energy—M tter Interactions 
in the Atmosphere Once 
Again 


The radiant flux reflected or emitted from the Earth’s 
surface once again enters the atmosphere, where it in- 
teracts with the various gases, water vapor, and partic- 
ulates. Thus, atmospheric scattering, absorption, 
reflection, and refraction influence the radiant flux 
once again before the energy is recorded by the remote 
sensing system. 


Energy—NM tter Interactions 
at the Sensor System 


Finally, energy—matter interactions take place when 
the energy reaches the remote sensing instrument. If an 
analog aerial camera is being used, then the radiance 
will interact with the camera filter, the optical glass 
lens, and finally the film emulsion with its very light- 
sensitive silver halide crystals. The emulsion must then 
be developed and printed before an analog hard copy 
is available for analysis. Rather than storing a latent 


CHAPTER 6 ELECTROMAGNETIC RADIATION PRINCIPLES AND RADIOMETRIC CORRECTION 205 


image on film, an optical—mechanical detector will dig- 
itally record the number of photons in very specific 
wavelength regions reaching the sensor. 


Correcting Remote Sensing 
Detector Error 


Ideally, the radiance recorded by a remote sensing sys- 
tem in various bands is an accurate representation of 
the radiance actually leaving the feature of interest 
(e.g., soil, vegetation, rocks, water, and/or urban land 
cover) on the Earth’s surface. Unfortunately, noise (er- 
ror) can enter the data-collection system at several 
points. For example, radiometric error in remotely 
sensed data may be introduced by the sensor system it- 
self when the individual detectors do not function 
properly or are improperly calibrated. Several of the 
more common remote sensing system—induced radio- 
metric errors are: 


¢ random bad pixels (shot noise), 

¢ line-start/stop problems, 

¢ line or column drop-outs, 

* partial line or column drop-outs, and 
¢ line or column striping. 


Sometimes digital image processing can recover the 
miscalibrated spectral information and make it rela- 
tively compatible with the correctly acquired data in 
the scene. Unfortunately, sometimes only cosmetic ad- 
justments can be made to compensate for the fact that 
no data of value were acquired. 


Random Bad Pixels (Shot Noise) 


Sometimes an individual detector does not record 
spectral data for an individual pixel. When this occurs 
randomly, it is called a bad pixel. When there are nu- 
merous random bad pixels found within the scene, it is 
called shot noise because it appears that the image was 
shot by a shotgun. Normally these bad pixels contain 
values of 0 or 255 (in 8-bit data) in one or more of the 
bands. Shot noise is identified and repaired using the 
following methodology. 


It is first necessary to locate each bad pixel in the band 
k dataset. A simple thresholding algorithm makes a 
pass through the dataset and flags any pixel (BV; ; x) 
having a brightness value of zero (assuming values of 0 
represent shot noise and not a real land cover such as 
water). Once identified, it is then possible to evaluate 
the eight pixels surrounding the flagged pixel, as shown 
below: 


col; co L coli+ 


The mean of the eight surrounding pixels is co mputed 
using Equation 6.23 and the value substituted for 
BV; ;;,in the corrected image: 


TOW; 


(6.23) 


8 
EAE 


i=1 


B ean ig Int 


This operation is performed for every bad (shot noise) 
pixel in the dataset. 


For example, Landsat Thematic Mapper band 7 imag- 
ery (2.08 — 2.35 Mm) of the Santee Delta is shown in 
Figure 6-20. It contains two pixels along a bad scan 
line with values of zero. The eight brightness values 
surrounding each bad pixel are enlarged and annotated 
in Figure 6-20b. The Landsat TM band 7 image after 
shot noise removal is shown in Figure 6-20c. 


Line or Column Drop-Outs 


An entire line containing no spectral information may 
be produced if an individual detector in a scanning sys- 
tem (e.g., Landsat 7 ETM7%) fails to function properly. 
If a detector in a linear array (e.g., SPOT 5, Geo-Eye-1, 
World View-2) fails to function, this can result in an en- 
tire column of data with no spectral information. The 
bad line or column is commonly called a /ine or column 
drop-out and contains brightness values equal to zero. 
For example, if one of the 16 detectors in the Landsat 
Thematic Mapper sensor system fails to function dur- 
ing scanning, this can result in a brightness value of 
zero for every pixel, j, in a particular line, 7 This /ine 
drop-out would appear as a completely black line in the 
band, k, of imagery. This is a serious condition because 
there is no way to restore data that were never ac- 
quired. However, it is possible to improve the visual in- 
terpretability of the data by introducing estimated 
brightness values for each bad scan line. 


It is first necessary to locate each bad line in the image 
dataset. A simple thresholding algorithm makes a pass 
through the dataset and flags any scan line having a 
mean brightness value at or near zero. Once identified, 
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Correction of Shot Noise 


a - = 4 


a. Landsat TM band 7 data of the Santee Delta with shot noise. 


b. Two pixels along a bad scan line with shot noise. c. Shot noise removed. 


FIGURE 6-20 a) Landsat Thematic Mapper band 7 (2.08 — 2.35 im) image of the Santee Delta in South Carolina. One of the 
16 detectors exhibits serious striping and the absence of brightness values at pixel locations along a scan line. b) An enlarged 
view of the bad pixels with the brightness values of the eight surrounding pixels annotated. c) The brightness values of the 
bad pixels after shot noise removal. This image was not destriped. Original imagery courtesy of NASA. 


it is then possible to collect the value for a pixel in the 


preceding line (BV; _ 1h) and the value for a pixel in BV gy oy BV ag ep 
the succeeding line (BV; | ;,) and assign the output BV; to Ch ete : (6.24) 


pixel (BV; ;,) in the drop-out line the average of these 
two brightness values: 
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This is performed for every pixel in a bad scan line. The 
result is an image consisting of interpolated data every 
nth line that is more visually interpretable than one 
with horizontal black lines running systematically 
throughout the entire image. This same cosmetic digi- 
tal image processing procedure can be applied to col- 
umn drop-outs produced by a linear array remote 
sensing system. 


Partial Line or Column Drop-Outs 


Occasionally an individual detector will function per- 
fectly along a scan line and then for some unknown 
reason it will not function properly for m columns. 
Then sometimes the detector functions properly again 
for the remainder of the scan line. The result is a por- 
tion of a scan line with no data. This is commonly re- 
ferred to as a partial line or partial column drop-out 
problem. This is a serious condition. It usually cannot 
be dealt with systematically because it occurs random- 
ly. If the portion of the image with the drop-out prob- 
lem is particularly important, then the analyst must 
manually correct the problem pixel by pixel. The ana- 
lyst must go into the dataset and compute the mean of 
the brightness values above and below each bad pixel 
and place the average in the bad pixel location. This is 
done for every pixel in the bad portion of the scan line. 


Line-Start Problems 


Occasionally, scanning systems fail to collect data at 
the beginning of a scan line, or they place the pixel 
data at inappropriate locations along the scan line. For 
example, all of the pixels in a scan line might be sys- 
tematically shifted just one pixel to the right. This is 
called a line-start problem. Also, a detector may 
abruptly stop collecting data somewhere along a scan 
and produce results similar to the line or column drop- 
out previously discussed. Ideally, when data are not 
collected, the sensor system would be programmed to 
remember what was not collected and place any good 
data in their proper geometric locations along the scan. 
Unfortunately, this is not always the case. For example, 
the first pixel (column 1) in band k on line 7 (1e., 
BYV;,;,;) might be improperly located at column 50 (i.e., 
BV;.50,x). If the line-start problem is always associated 
with a horizontal bias of 50 columns, it can be correct- 
ed using a simple horizontal adjustment. However, if 
the amount of the line-start displacement is random, it 
is difficult to restore the data without extensive human 
interaction on a line-by-line basis. A considerable 
amount of MSS data collected by Landsats 2 and 3 ex- 
hibits line-start problems. 


Figure 6-21a depicts predawn thermal infrared remote 
sensor data of the Savannah River obtained on March 
31, 1981. It contains several line-start problems that 


are visually apparent. Seven lines with line-start prob- 
lems (which contain good radiant energy information) 
were adjusted horizontally (to the left) just one pixel to 
be in their proper position (Figure 6-21b). Accurate 
linear measurements can now be made from the ad- 
justed data. In addition, the image is now much more 
visually appealing. 


N-Line Striping 


Sometimes a detector does not fail completely, but sim- 
ply goes out of radiometric adjustment. For example, a 
detector might record spectral measurements over a 
dark, deep body of water that are almost uniformly 20 
brightness values greater than the other detectors for 
the same band. The result would be an image with sys- 
tematic, noticeable lines that are brighter than adjacent 
lines. This is referred to as n-line striping. The malad- 
justed line contains valuable information, but should 
be corrected to have approximately the same radiomet- 
ric scale as the data collected by the properly calibrated 
detectors associated with the same band. 


To repair systematic n-line striping, it is first necessary 
to identify the miscalibrated scan lines in the scene. 
This is usually accomplished by computing a histo- 
gram of the values for each of the detectors that col- 
lected data over the entire scene (ideally, this would 
take place over a homogeneous area, such as a body of 
water). If one detector’s mean or median is significant- 
ly different from the others, it is probable that this de- 
tector is out of adjustment. Consequently, every line 
and pixel in the scene recorded by the maladjusted de- 
tector may require a bias (additive or subtractive) cor- 
rection or a more severe gain (multiplicative) 
correction. This type of n-line striping correction a) ad- 
justs all the bad scan lines so that they have approxi- 
mately the same radiometric scale as the correctly 
collected data and b) improves the visual interpretabili- 
ty of the data. It looks better. 


For example, Figure 6-22a presents a Landsat TM col- 
or composite (RGB = bands 4,3,2) of the Santee River 
delta in South Carolina obtained in 1984. The area 
consists primarily of salt marsh. The image exhibits se- 
rious striping every 16 lines. A pass was made through 
the dataset to identify the maladjusted lines. A gain 
and a bias were computed and applied to each affected 
line. Figure 6-22b depicts the same three band color 
composite after destriping. The striping is noticeably 
subdued, although close inspection reveals some resid- 
ual striping. It is usually difficult to remove all traces of 
striping. 


A 35-band hyperspectral 2% 2 m spatial resolution da- 
taset of the Mixed Waste Management Facility on the 
Savannah River Site in Aiken, SC, was acquired on 
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Line-start Problems 


a. Predawn thermal infrared imagery of the 
Savannah River with line-start problems. 


b. Seven line-start problem lines were 
translated one column to the left. 


Fl QURE 62 1 a) Infrared imagery of the Four Mile Creek thermal effluent plume entering the Savannah River on March 31, 
1981. Seven lines with line-start problems are present. b) The result of translating each of the line-start problem lines one col- 
umn to the left. The line-start problem was systematic in that offending lines only needed to be adjusted by one column. The 
line-start problem was unsystematic because not all of the lines in the dataset exhibited the line-start problem. Therefore, it 
was necessary to manually correct (using digital image processing) each of the line-start problem lines. 


July 31, 2002 (Figure 6-23a). The level, clay-capped 
hazardous waste site fields are covered with Bahia 
grass or Centipede grass. A horizontal spectral profile 
through band 5 (red; centered at 633 nm) reveals good 
radiometric integrity along the columns of the dataset 
(Figure 6-24a). Unfortunately, a vertical spectral pro- 
file through the waste site exhibits a sawtooth pattern 
indicative of striping (Figure 6-24b). Using the previ- 
ously described methodology, a gain and a bias were 
identified and applied to each offending fourth line in 
the dataset. The result is the radiometrically adjusted 
band 5 image shown in Figure 6-23cd. Examination of 
a vertical spectral profile through the corrected 
(destriped) dataset reveals that the sawtooth striping 
was greatly minimized but not completely removed 
(Figure 6-24c). The visual appearance of the band 5 
data is noticeably improved. 


Remote Sensing 
Atmospheric Correction 


Even when the remote sensing system is functioning 
properly, radiometric error may be introduced into the 


remote sensor data. The two most important sources 
of environmental attenuation are: 1) atmosphere atten- 
uation caused by scattering and absorption in the at- 
mosphere, and 2) topographic attenuation. However, it 
is important to first consider that it may not be neces- 
sary to atmospherically correct the remote sensor data 
for all applications. The decision to perform an atmo- 
spheric correction is a function of the nature of the 
problem, the type of remote sensing data available, the 
amount of in situ historical and/or concurrent atmo- 
spheric information available, and how accurate the 
biophysical information to be extracted from the re- 
mote sensing data must be (He and Chen, 2014). 


Unnecessary Atmospheric Correction 


Sometimes it is possible to ignore atmospheric effects 
in remote sensor data completely (Cracknell and 
Hayes, 1993; Song et al., 2001). For example, atmo- 
spheric correction is not always necessary for certain 
types of classification and change detection. Theoreti- 
cal analysis and empirical results indicate that only 
when training data from one time or place must be ex- 
tended through space and/or time is atmospheric cor- 
rection necessary for image classification and many 


CHAPTER 6 ELECTROMAGNETIC RADIATION PRINCIPLES AND RADIOMETRIC CORRECTION 209 


ALL in Landsat Thematic Mapper Imagery 


a. Landsat TM color composite of the 
Santee River delta in South Carolina. 


FIG 


RE 6 


b. Landsat TM data destriped. 


22 a) A Landsat Thematic Mapper image color composite (RGB = bands 4,3,2) of the Santee River delta in South 


Carolina in 1984. The area consists primarily of smooth cordgrass salt marsh (Spartina alterniflora). The image exhibits serious 
striping every 16 lines which have been highlighted by enhancing the contrast of the original TM dataset. A pass was made 
through the original dataset to identify the maladjusted lines. A gain and a bias were computed and applied to each affected 
line. b) The same study area is displayed after the data were destriped. Original imagery courtesy of NASA. 


types of change detection (Song et al., 2001). For ex- 
ample, it is not generally necessary to perform atmo- 
spheric correction on a single date of remotely sensed 
data that will be classified using a maximum likelihood 
classification algorithm (discussed in Chapter 9). As 
long as the training data from the image to be classified 
have the same relative scale (corrected or uncorrected), 
atmospheric correction has little effect on classification 
accuracy (Song et al., 2001). 


For example, consider land-cover classification using a 
single date of Landsat Thematic Mapper data. Ray- 
leigh and other types of scattering normally add 
brightness to the visible bands (400 — 700 nm). Atmo- 
spheric absorption is the most important factor that re- 
duces the brightness values of pixels in the near- and 
middle-infrared region (700 — 2,400 nm). Fortunately, 
the Landsat TM near- and middle-infrared bandwidths 
were carefully chosen to minimize the effects of atmo- 
spheric absorption. Therefore, if a single date of Land- 
sat TM data is atmospherically corrected, it is likely 


that the primary effect will be a simple bias adjustment 
applied separately to each band. This action would ad- 
just the minimum and maximum values of each band 
downward. Training class means extracted from the 
single-date image would change but the training class 
variance—covariance matrices should remain invariant. 
Therefore, the actual information content used in the 
maximum likelihood classification of the dataset would 
remain unchanged. 


This logic also applies to certain types of change detec- 
tion. For example, consider a change detection study 
involving two dates of imagery. If the two dates of im- 
agery are analyzed independently using a standard 
classification algorithm and the resultant classification 
maps are compared using post-classification change 
detection logic (discussed in Chapter 12), there is no 
need to atmospherically correct the individual dates of 
remote sensor data. Similarly, atmospheric correction 
is not necessary when performing multidate composite 
image change detection (Jensen et al., 1993) where the 
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Striped Hyperspectral Imagery of the Savannah River Site 


a. Original band 10 radiance. 


c. Destriped band 10 radiance. 


b. Original band 10 magniXed. 


d. Destriped band 10 magnied. 


FIGURE 6-23 a) Original band 10 radiance (W m® sr) data from a GER DAIS 3715 hyperspectral dataset of the Mixed 
Waste Management Facility on the Savannah River Site near Aiken, SC. The subset is focused on a clay-capped hazardous 
w esitew edwithBh agrasd Centipede grass. The 35-bandd tae wsb aid at2X 2mp atialresbutio . 
The radiance values along the horizontal (X) and vertical (Y) p 6 ile are sm marizd in Fig e 6-24. b) An enlargement of the 
band 10 data. c) The band 10 data after destriping. d) An enlargement of the destriped data (Jensen et al., 2003). 


change detection algorithm identifies change classes 
based on an analysis of all bands on both dates being 
placed in a single dataset (e.g., four TM bands from 
date 1 and four TM bands from date 2 are placed in a 
single eight-band dataset). 


The general principle is that atmospheric correction is 
not necessary as long as the training data are extracted 
from the image (or composite image) under investiga- 
tion and are not imported from another image ob- 
tained at another place or time. 
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FIGURE 6-24 a) The radiance values along the horizontal (X) profile of the original band 10 radiance values in Figure 6-23. b) 
The radiance values along the vertical (Y) profile of the original band 10 radiance values in Figure 6-23. c) The radiance values 
along the vertical (Y) profile of the destriped band 10 radiance values. Note the reduction of the saw-toothed pattern in the 


destriped data (Jensen et al., 2003). 


Necessary Atmospheric Correction 


Sometimes it is essential that the remotely sensed data 
be atmospherically corrected. For example, it is usually 
necessary to atmospherically correct the remote sensor 
data if biophysical parameters are going to be extract- 
ed from water bodies (e.g., chlorophyll a, suspended 
sediment, temperature) or vegetation (e.g., biomass, 


leaf-area-index, chlorophyll, percent canopy closure). 
If the data are not corrected, the subtle differences in 
reflectance (or emittance) among the important con- 
stituents may be lost. Furthermore, if the biophysical 
measurements extracted from one image (e.g., bio- 
mass) are to be compared with the same biophysical in- 
formation extracted from other images obtained on 
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different dates, then it is usually essential that the re- 
mote sensor data be atmospherically corrected. 


For example, consider the case of the normalized dif- 
ference vegetation index (NDVI) derived from Landsat 
Thematic Mapper (TM) band 3 (red) and band 4 
(near-infrared) data: 


Pnir Pred 
Prip * Pred 


NDVI = 


which is used routinely to measure vegetation biomass 
and functional health in many decision-support sys- 
tems such as the Africa Famine Early Warning System 
and Livestock Early Warning System (Jensen et al., 
2002). Erroneous NDVI estimates can result in the loss 
of livestock and human life. Contributions from the at- 
mosphere to NDVI are significant and can amount to 
50% or more over thin or broken vegetation cover. 
Therefore, there is significant interest in removing the 
deleterious effects of the atmosphere in remotely 
sensed data that are used to compute NDVI estimates. 
The simple ratio (SR) vegetation index (TM4/TM3 for 
Landsat TM data) is also contaminated by the atmo- 
sphere (Song et al., 2001). 


Great emphasis is being placed on the development of 
algorithms that can extract biophysical information 
from remotely sensed data for local, regional, and 
global applications. These data are then placed in vari- 
ous models and decision-support systems to monitor 
global processes and hopefully improve the quality of 
life. Consequently, great emphasis is being placed on 
the ability to derive measurements from the remotely 
sensed data that are accurate across vast distances and 
through time. Signature extension through space and 
time is becoming more important. The only way to ex- 
tend signatures through space and time is to atmo- 
spherically correct each individual date of remotely 
sensed data. 


Types of Atmospheric Correction 


There are several ways to atmospherically correct re- 
motely sensed data. Some are relatively straightfor- 
ward while others are complex, being founded on 
physical principles and requiring a significant amount 
of information to function properly. This discussion 
will focus on two major types of atmospheric correc- 
tion: 


¢ absolute atmospheric correction, and 
¢ relative atmospheric correction. 


There are various methods that can be used to achieve 
absolute or relative atmospheric correction. The fol- 


lowing sections identify the logic, some of the algo- 
rithms, and problems associated with each 
methodology. 


1. It is possible to use a model atmosphere to correct 
the remotely sensed data. An assumed atmosphere 
is calculated based on the time of year, altitude, 
latitude, and longitude of the study area. This 
approach may be successful when atmospheric 
attenuation is relatively small compared with the 
signal from the terrain being remotely sensed 
(Cracknell and Hayes, 1993). 

2. The use of a model atmosphere in conjunction with 
in situ atmospheric measurements acquired at the 
time of remote sensor data acquisition is even 
better. Sometimes, the in situ data can be provided 
by atmospheric sounding instruments found 
onboard the sensor platform. The atmospheric 
model may then be fine-tuned using the local 
condition information. This is referred to as 
absolute radiometric correction, and an example is 
provided later in the chapter. 

3. Minimization of atmospheric attenuation is 
sometimes possible using multiple looks at the 
same object from different vantage points (e.g., 
fore and aft) or by looking at the same object using 
multiple bands of the spectrum. The goal is to try 
to have the information from the multiple looks or 
multiple bands cancel out the atmospheric effects. 
The multiple-look method suffers from the fact 
that the atmospheric paths for the multiple looks 
(e.g., fore and aft) may not be the same. 
Theoretically, the band-cancellation method 
should be capable of providing good results 
because it is using identical atmospheric paths for 
the channels that are being compared. This is called 
relative radiometric correction, and an image 
normalization example will be provided. 


Absolute Radiometric Correction of 
Atmospheric Attenuation 


Solar radiation (£,, ) is largely unaffected as it travels 
through the vacuum of space (Figure 6-25). When it in- 
teracts with the Earth’s atmosphere, however, it is se- 
lectively scattered and absorbed. The sum of these two 
forms of energy loss is called atmospheric attenuation. 
Serious atmospheric attenuation may 1) make it diffi- 
cult to relate hand-held in situ spectroradiometer mea- 
surements of materials with remote measurements of 
these same materials, 2) make it difficult to extend 
spectral signatures through space and time, and 3) have 
an impact on classification accuracy within a scene if 
atmospheric attenuation varies significantly through- 
out the image (Cracknell and Hayes, 1993; Kaufman et 
al., 1997). 
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Various Paths of 
Radiance Received by 
a Remote Sensing System 
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FIGURE 6-25 Radiance (L;) from paths 1, 3, and 5 contains intrinsic valuable spectral information about the target of interest. 
Conversely, the path radiance (L,) from paths 2 and 4 includes diffuse sky irradiance or radiance from neighboring areas on 
the ground. This path radiance generally introduces unwanted radiometric noise in the remotely sensed data and compli- 


cates the image interpretation process. 


The general goal of absolute radiometric correction is to 
turn the digital brightness values recorded by a remote 
sensing system into scaled surface reflectance values 
(Du et al., 2002). These values can then be compared 
or used in conjunction with scaled surface reflectance 
values obtained anywhere else on the planet. 


A considerable amount of research has been carried 
out to address the problem of correcting images for at- 
mospheric effects. These efforts have resulted in a num- 
ber of atmospheric radiative transfer codes (models) 
that can provide realistic estimates of the effects of at- 
mospheric scattering and absorption on imagery (e.g., 
Kruse, 2004; Alder-Golden et al., 2005; Gao et al., 
2009; Agrawal and Sarup, 2011; Richter and Schlapfer, 
2014). Once these effects have been identified for a spe- 
cific date of imagery, each band and/or pixel in the 


scene can be adjusted to remove the effects of scatter- 
ing and/or absorption. The image is then considered to 
be atmospherically corrected. 


Unfortunately, the application of these codes to a spe- 
cific scene and date also requires knowledge of both 
the sensor spectral profile and the atmospheric proper- 
ties at the same time. Atmospheric properties are diffi- 
cult to acquire even when planned. For most historic 
satellite data, they are not available. Even today, accu- 
rate scaled surface reflectance retrieval is not opera- 
tional for the majority of satellite image sources used 
for land-cover change detection. One exception is NA- 
SA’s Moderate Resolution Imaging Spectroradiometer 
(MODIS), for which surface reflectance products are 
available. Nevertheless, we will proceed with a very 
general discussion of the important issues associated 
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with absolute atmospheric correction and then provide 
examples of how absolute radiometric correction is 
performed. 


Target and Path Radiance 

Ideally, the radiance (L) recorded by the camera or de- 
tector is a true function of the amount of radiance 
leaving the target terrain within the instantaneous- 
field-of-view (IFOV) at a specific solid angle, as previ- 
ously discussed. Unfortunately, other radiant energy 
may enter into the field of view from various other 
paths and introduce confounding noise into the remote 
sensing process. Therefore, additional radiometric vari- 
able definitions are needed to identify the major sourc- 
es and paths of this energy (Green, 2003). The 
variables are summarized in Table 6-5. The various 
paths and factors that determine the radiance reaching 
the remote sensing detector(s) are summarized in Fig- 
ure 6-25, including: 


¢ Path 1 contains the spectral solar irradiance (£, e 
that was attenuated very little before illuminating 
the terrain within the IFOV. Notice that we are 
interested in the solar irradiance from a specific 
solar zenith angle (0, ) and that the amount of irra- 
diance reaching the terrain is a function of the 
atmospheric transmittance at this angle (7, ). If all 
of the irradiance makes it to the ground, then the 
atmospheric transmittance (To, ) equals one. If none 
of the irradiance makes it to the ground, then the 
atmospheric transmittance is zero. 


¢ Path 2 contains spectral diffuse sky irradiance (£, ) 
that never even reaches the Earth’s surface (the tar- 
get study area) because of scattering in the atmo- 
sphere. Unfortunately, such energy is often scattered 
directly into the IFOV of the sensor system. As pre- 
viously discussed, Rayleigh scattering of blue light 
contributes much to this diffuse sky irradiance. That 
is why the blue band image produced by a remote 
sensor system is often much brighter than any of the 
other bands. It contains much unwanted diffuse sky 
irradiance that was inadvertently scattered into the 
IFOV of the sensor system. Therefore, if possible, 
we want to minimize its effects. Green (2003) refers 
to the quantity as the upward reflectance of the 
atmosphere (£ du, ). 


¢ Path 3 contains energy from the Sun that has under- 
gone some Rayleigh, Mie, and/or nonselective scat- 
tering and perhaps some absorption and re-emission 
before illuminating the study area. Thus, its spectral 
composition and polarization may be somewhat dif- 
ferent from the energy that reaches the ground from 
path 1. Green (2003) refers to this quantity as the 
downward reflectance of the atmosphere (£ dd, ). 


TABLE 6-5 Radiometric variables used in remote sensing. 


Radiometric Variables 


E, = solar irradiance at the top of the atmosphere (W m~) 


E, = spectral solar irradiance at the top of the atmosphere 
a 


(W m2 m7!) 
Ey = diffuse sky irradiance (W m-“) 
Ey = spectral diffuse sky irradiance (W m2 kim") 
a 
E iu = the upward reflectance of the atmosphere 
h 


Edd = the downward reflectance of the atmosphere 
a 


E = global irradiance incident on the surface (W m2) 

= = spectral global irradiance on the surface (W m~? Xm") 
T  =normal atmospheric optical thickness 

To = atmospheric transmittance at an angle 9 to the zenith 
0, = solar zenith angle 


0, = view angle of the satellite sensor (or scan angle) 
[L =cos 9 
Py = surface target reflectance at a specific wavelength 


Py = reflectance from a neighboring area 


ib. = total radiance at the sensor (W m~ sr’) 
Ib eS total radiance from the target of interest toward the 
= 
sensor (W m~ sr_) 
L._ =intrinsic radiance of the target (W m- sr!) (i.e., what a 


hand-held radiometer would record on the ground without 
intervening atmosphere) 


Ly = path radiance from multiple scattering (W m~ sr!) 


¢ Path 4 contains radiation that was reflected or scat- 
tered by nearby terrain ( Pr. ) covered by snow, con- 
crete, soil, water, and/or vegetation into the IFOV of 
the sensor system. The energy does not actually illu- 
minate the study area of interest. Therefore, if possi- 
ble, we would like to minimize its effects. 


¢ Path 5 is energy that was also reflected from nearby 
terrain into the atmosphere, but then scattered or 
reflected onto the study area. 


Therefore, for a given spectral interval in the electro- 
magnetic spectrum (e.g., A, to A, could be 0.6 — 0.7 
Xm or red light), the total solar irradiance reaching the 
Earth’s surface, E a is an integration of several impor- 
tant components: ~ 
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dy 
Ey = | (E,,Tp,0088, + £4) (W mm"). (6.25) 
ry 
It is a function of the spectral solar irradiance at the 
top of the atmosphere (£, ) multiplied by the atmo- 
spheric transmittance ( To. ) at a certain solar zenith an- 


gle (0,) plus the contribution of spectral diffuse sky 
irradiance (£ dy ). 


Only a small amount of this irradiance is actually re- 
flected by the terrain in the direction of the satellite 
sensor system. If we assume the surface of Earth is a 
diffuse reflector (a Lambertian surface), the total 
amount of radiance exiting the target study area (L7) 
toward the sensor is: 


he 
Re 

Ly = =| py, To (Eo, 79,0080, + By, dr. 
Ay 


(6.26) 


The average surface target reflectance (p, ) is included 
because the vegetation, soil, and water within the 
IFOV selectively absorb some of the incident energy. 
Therefore, not all of the energy incident to the IFOV 
(E ) leaves the IFOV. In effect, the terrain acts like a 
filter, selectively absorbing certain wavelengths of light 
while reflecting others. Note that the energy exiting the 
terrain is at an angle (0,,), requiring the use of an at- 
mospheric transmittance factor To once again. 


It would be wonderful if the total radiance recorded by 
the sensor, Ls, equaled the radiance returned from the 
target study area of interest, L;. Unfortunately, L, # 
Ly because there is some additional radiance from dif- 
ferent paths that may fall within the IFOV of the sen- 
sor system detector (Figure 6-25). This is often called 
path radiance, Lp. Thus, the total radiance recorded by 
the sensor becomes: 


Ly=Lpt+ Lp (Wm? sr"). (6.27) 


We see from Equation 6.27 and Figure 6-25 that the 
path radiance (Lp) is an intrusive (bad) component of 
the total amount of radiance recorded by the sensor 
system (L,). It is composed of radiant energy primarily 
from the diffuse sky irradiance (£,)) from path 2 as well 
as the reflectance from nearby ground areas Pr, from 
path 4. Path radiance introduces error to the remote 
sensing data-collection process. It can impede our abil- 
ity to obtain accurate spectral measurements. 


A great deal of research has gone into developing meth- 
ods to remove the contribution of path radiance (Lp). 
Methods for computing path radiance are summarized 


in Richards (2013). Radiative transfer model programs 
such as MODTRAN, Second Simulation of the Satel- 
lite Signal in the Solar Spectrum (6S) and others may 
be used to predict path radiance on a particular day for 
a particular study area (e.g., Alder-Golden et al., 1999, 
2005; Matthew et al., 2000; Gao et al., 2009; Richter 
and Schlapfer, 2014). Such information can then be 
used to remove the path radiance (Lp) contribution to 
the remote sensing signal (L;). 


Atmospheric Transmittance 

To understand how to remove atmospheric attenua- 
tion, it is useful to review the fundamental mechanisms 
of atmospheric scattering and absorption related to at- 
mospheric transmittance. In the absence of atmo- 
sphere, the transmittance of solar radiant energy to the 
ground would be 100%. However, because of absorp- 
tion and scattering, not all of the radiant energy reach- 
es the ground. The amount that does reach the ground, 
relative to that for no atmosphere, is called transmit- 
tance. Atmospheric transmittance (T)) may be com- 
puted as: 


= ge 7/ (eos 0) (6.28) 


T 
where t is the normal atmospheric optical thickness 
and 0 can represent either 0, or 0, in Figure 6-25 (Le., 
the ability of the atmosphere to transmit radiant flux 
from the Sun to the target, 7, 9,7 OF the ability of the at- 
mosphere to transmit radiant flux from the target to 
the sensor system, T 0, ). The optical thickness of the 
atmosphere at certain wavelengths, t(4), equals the 
sum of all the attenuating coefficients, which are made 
up primarily of Rayleigh scattering, t,,, Mie scatter- 
ing, T 5a and selective atmospheric absorption, t, : 


tA) = t+ i +h. (6.29) 


where Tt = THo + To, + To, + To, 

As previously discussed, Rayleigh scattering of gas mol- 
ecules occurs when the diameter of the gas molecule (d) 
is less than the size of the incident wavelength (d <i) 
and is inversely proportional to the fourth power of the 
wavelength, 1/ a4” Aerosol (Mie) scattering occurs 
when d = X and is nana a function of water vapor, 
dust, and other aerosol particles in the atmosphere. Se- 
lective absorption of radiant energy in the atmosphere 
is wavelength-dependent. Most of the optical region 
from 0.4 to 1.0 Xm is dominated by water and ozone ab- 
sorption attenuation. Atmospheric absorption by water 
vapor and other gases in the atmosphere mostly affects 
radiation of wavelengths > 0.8 Xm. Thus, atmospheric 
scattering may add brightness to, whereas atmospheric 
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absorption may subtract brightness from, landscape 
spectral measurements. 


Diffuse Sky Irradiance 

Path 1 in Figure 6-25 contains radiation that followed 
a direct path from the Sun, to the target study area 
within the IFOV, to the remote sensing detector. Un- 
fortunately, some additional scene irradiance comes 
from scattered skylight. Path 3 in Figure 6-25 contains 
radiation that has undergone some scattering before il- 
luminating the study area. Thus, its spectral composi- 
tion may be somewhat different. Similarly, path 5 
contains energy that was reflected from a nearby area 
on the ground into the atmosphere and then scattered 
once again onto the study area. The total diffuse sky ir- 
radiance at the pixel is E). 


It is now possible to determine how these atmospheric 
effects (transmittance, diffuse sky irradiance, and path 
radiance) affect the radiance measured by the remote 
sensing system. First, however, because the bandwidths 
used in remote sensing are relatively narrow (e.g., 0.5 
to 0.6 Xm), it is possible to restate Equations 6.25 and 
6.26 without the integral. For example, the total irradi- 
ance at the Earth’s surface may be written: 


E. = Egan, cs Q AAT+E,, (6.30) 


where E,, 1s the average spectral irradiance in the 
band interval AA = 4, —2,. The total radiance trans- 
mitted through the atmosphere toward the sensor (L7) 
becomes: 


1 
Ly = PT o (Eoante,©os Q AA+£,). (6.31) 


The total radiance reaching the sensor then becomes: 


1 
Lg = =pTg (Egan 008 9,AX + Eg) + Lp, (6.32) 


which may be used to relate brightness values in re- 
motely sensed data to measured radiance, using the 
equation: 


Ly = (Kx BY, 5 y)+Lnin> (6.33) 


where 


K=radiance per bit of sensor count rate = 
(Lmax ~ Lyin)! Cmax 
BV, = brightness value of a pixel 
Cmax = Maximum value in dataset (e.g., 8-bit = 255) 


Lmax = tadiance measured at detector saturation 


(W ic? sr!) 
Lmin = lowest radiance measured by a detector 


(W m~ sr!) 


The Lyin and Lyx Values are available for all the ma- 
jor remote sensing systems. 


Atmospheric Correction Based on Radiative 
Transfer Modeling 


Most current radiative transfer-based atmospheric cor- 
rection algorithms can compute much of this same in- 
formation if a) the user provides fundamental 
atmospheric characteristic information to the program, 
or b) certain atmospheric absorption bands are present 
in the remote sensing dataset. For example, most radia- 
tive transfer-based atmospheric correction algorithms 
require that the user provide: 


¢ latitude and longitude of the remotely sensed image 
scene, 


¢ the date and precise time that the remote sensing 
data were collected, 


* image acquisition altitude (e.g., 20 km AGL), 
¢ mean elevation of the scene (e.g., 200 m ASL), 


* an atmospheric model (e.g., mid-latitude summer, 
mid-latitude winter, tropical), 


* radiometrically calibrated image radiance data (Le., 
the data must be in the form W m? Xm’! sr), 


¢ information about each specific band (i.e., its mean 
and full-width at half-maximum (FWHM), and 


¢ local atmospheric visibility at the time of remote 
sensing data collection (e.g., 10 km, obtained from a 
nearby airport if possible). 


These parameters are then input to the atmospheric 
model selected (e.g., mid-latitude summer) and used to 
compute the absorption and scattering characteristics 
of the atmosphere at the instance of remote sensing 
data collection. These atmospheric characteristics are 
then used to invert the remote sensing radiance to 
scaled surface reflectance. Many of these atmospheric 
correction programs derive the scattering and absorp- 
tion information they require from robust atmosphere 
radiative transfer code such as MODTRAN 4+ (AI- 
der-Golden et al., 1999; 2005) or Second Simulation of 
the Satellite Signal in the Solar Spectrum (6S) (Ver- 
mote et al., 1997, 2002; Gao et al., 2009). 


Numerous atmospheric correction algorithms have 
been developed based on the use of radiative transfer 
principles (Gao et al., 2009). Several of the most im- 
portant are ACORN, QUAC, FLAASH, and ATCOR. 
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It is instructive briefly to review some of their charac- 
teristics. 


« ACORN—Atmospheric CORrection Now is based 
on MODTRAN 4 radiative transfer code (San and 
Suzen, 2010; ImSpec ACORN, 2014). It performs 
atmospheric correction of both multispectral and 
hyperspectral imagery in the region from 250 — 
2,500 nm. It is designed to work with all airborne 
and spaceborne calibrated remote sensing systems 
such as Hyperion, ASTER, Landsat ETM’, 
AVIRIS, SPOT, GeoEye-1, etc. It defines the rela- 
tionship from contributions of the exo-atmospheric 
solar source, a homogenous plane parallel atmo- 
sphere, and the surface with respect to the radiance 
measured by an Earth-looking remote sensor 
(ImSpec ACORN, 2014). Kruse (2004) suggests that 
all model-based atmospheric correction algorithms 
such as ACORN and FLAASH basically use the 
radiative transfer model shown below, though each 
model uses a slightly different version: 


Ly = [E,,T, p, cos(8)] +Lp, (6.34) 


where L. is the total radiance measured by the sen- 
sor, E, is the solar spectral irradiance at the top of 
the atmosphere, 7, is the total atmospheric trans- 
mittance, p, is the spectral reflectance of the surface, 
and Lp is the path radiance. 


Current atmospheric correction programs assume 
that the Earth’s surface is horizontal and has Lam- 
bertian reflectance. This is because for real data we 
typically don’t have enough information to make the 
topographic correction. The end result is called 
scaled surface reflectance or apparent reflectance. The 
scaled surface reflectance can be converted to surface 
reflectance if the surface topography is known 
(Kruse, 2004). The principal outputs of ACORN are 
a water vapor image and scaled surface reflectance 
data. 


For example, consider the hyperspectral data of the 
Savannah River Site that was atmospherically cor- 
rected and transformed from radiance to scaled sur- 
face reflectance using ACORN. Figure 6-26a depicts 
the original radiance (in W m” ? nm” sr hy of a pixel of 
loblolly pine (Pinus taeda) in 35 bands of GER DAIS 
3715 hyperspectral imagery from 509 nm to 2,365 
nm. Figure 6-26b is the scaled surface reflectance of 
the same pixel. The Loblolly pine pixel now appears 
as it should, with a slight peak of reflectance in the 
green at approximately 566 nm, chlorophyll absorp- 
tion in the red centered at approximately 682 nm, and 
increased near-infrared reflectance (Jensen et al., 
2003). 
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FIGURE 6-26 a) Calibrated radiance data of a Loblolly Pine 
pixel on the Savannah River Site recorded by the GER DAIS 
3715 hyperspectral remote sensing system. The 35 bands 
range from 509 — 2,365 nm. b.) The same pixel transformed 
into scaled surface reflectance using the radiative transfer At- 
mospheric CORrection Now (ACORN) algorithm (based on 
Jensen et al., 2003). 


¢ FLAASH—ENVI’s Fast Line-of-sight Atmospheric 
Analysis of Spectral Hypercubes was developed by 
the Air Force Phillips Laboratory, Hanscom AFB 
and Spectral Sciences, Inc. FLAASH is an atmo- 
spheric correction program that corrects remote 
sensor data in the region from 400 — 3,000 nm (Exe- 
lis QUAC and FLAASH, 2009; San and Suzen, 
2010; Exelis Atmospheric Correction, 2014). 
FLAASH can be used to atmospherically correct 
data from hyperspectral sensors such as HyMAP, 
AVIRIS, HYDICE, HYPERION, Probe-1, CASI, 
and AISA and multispectral sensors such as 
ASTER, IRS, Landsat, RapidEye, and SPOT. 


FLAASH uses MODTRAN simulations of spectral 
radiance computed for various atmospheric, water 
vapor, and viewing conditions (solar angles) over a 
range of surface reflectances to establish lookup ta- 
bles for the atmospheric parameters of column water 
vapor, aerosol type, and visibility for subsequent use 
(Kruse, 2004; Alder-Golden et al., 2005). The 1.13 Km 
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AVIRIS Image of a Part of Jasper Ridge Biological Preserve 
in San Mateo County, CA, collected on April 3, 1998 


‘ Mga ina. Se 


a. Natural color composite consisting of red (band 28 centered on 636 nm), 
green (band 19 centered on 547 nm), and blue (band 10 centered on 458 nm). 
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FIGURE 627 a) AVIRIS 20 X 20 m hyperspectral image consisting of 224 bands (AVIRIS data courtesy of NASA Jet Propul- 
sion Lab). b) Uncorrected spectral profile of a pixel of concrete at column 369 and row 179 (value scale = radiance). ENVI 
FLAASH atmospherically corrected spectral profile of the same pixel (value scale = scaled percent reflectance X 1,000). 
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water band is used to estimate water vapor, and a ra- 
tio of in-band and out-of-band radiance values al- 
lows estimation of absorption band depths for a 
range of water vapor column densities. FLAASH 
also derives pressure altitudes by applying the same 
method to the oxygen 0.762 Km absorption band. 
The radiance spectra are extracted from the hyper- 
spectral data and compared against the MOD- 
TRAN lookup tables on a pixel-by-pixel basis to 
determine scaled surface reflectance. FLAASH of- 
fers the additional option of correcting for light scat- 
tered from adjacent pixels (Alder-Golden et al., 
1999; Matthew et al., 2003). It also produces a col- 
umn water vapor image, a cloud map, and a visibility 
value for the scene (Felde et al., 2003; Kruse, 2004; 
Exelis FLAASH Multispectral, 2014). 


An example of the application of FLAASH atmo- 
spheric correction to AVIRIS hyperspectral data is 
shown in Figure 6-27. The 20 X 20 m AVIRIS data of 
a part of the Jasper Ridge Biological Preserve con- 
sists of 224 bands and was acquired on April 3, 1998. 
A pixel centered on the concrete freeway at 366,179 
is highlighted in Figure 6-27a. The uncorrected spec- 
tral profile of this pixel is shown in Figure 6-27b. 
Note the impact of various atmospheric constituents 
such as oxygen, water vapor, and carbon dioxide on 
the at-sensor radiance spectral profile. The FLAASH 
atmospherically-corrected pixel exhibits normal 
scaled percent reflectance characteristics (Figure 6- 
27b) with high reflectance throughout the blue, 
green, red, and near-infrared portion of the spectrum 
as expected (e.g., refer to the hand-held spectroradi- 
ometer reflectance measurement of concrete shown 
in Figure 6-17). 


* QUAC—ENVI’s Quick Atmospheric Correction 
(QUAC) is a visible-near infrared through short- 
wave infrared (VNIR-SWIR) atmospheric correc- 
tion method that can be used for both multispectral 
and hyperspectral imagery (Figure 6-28) (Bernstein 
et al., 2008; Exelis QUAC, 2014; Exelis Atmospheric 
Correction, 2014). Unlike ACORN and FLAASH, 
it determines atmospheric compensation parameters 
from the information contained within the image 
such as pixel endmember spectra (ie., pure, 
unmixed pixels in the scene), without ancillary 
information (Figure 6-28). QUAC performs a more 
approximate atmospheric correction than FLAASH 
or other physics-based first-principles methods, gen- 
erally producing reflectance spectra within approxi- 
mately —15% of the physics-based approaches. 
QUAC is based on the empirical finding that the 
average reflectance of a collection of diverse mate- 
rial spectra, such as the endmember spectra in a 
scene, is essentially scene-independent. QUAC also 
allows for any view or solar elevation angle. Should 


Quick Atmospheric Correction (QUAC) 
Flow Diagram 


Baseline Determination 
(Darkest Channel/Offset) 


Baseline Subtraction 


Bright Spectra Filter 


Gain 
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Apply Gain and Offset to 
Retrieve ReXectance Data 


FIGURE 6-28 Quick Atmospheric Correction (QUAC) 
processing flow (adapted from Exelis QUAC and FLAASH, 
2014.). 


a sensor not have proper radiometric or wavelength 
calibration, or the solar illumination intensity be 
unknown (such as when a cloud deck is present), 
this approach still allows the retrieval of reasonably 
accurate reflectance spectra. QUAC provides (Exelis 
QUAC, 2014): 


- atmospheric correction of multispectral and hyper- 
spectral data in the region from 400 — 2,500 nm. 


- atmospheric correction of the following types of re- 
mote sensor data: AISA, ASAS, AVIRIS, CAP AR- 
CHER, COMPASS, HYCAS, HYDICE, HyMap, 
Hyperion, IKONOS, Landsat Thematic Mapper, 
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LASH, MASTER, MODIS, MTI, 
RGB, and unknown sensor types. 


QuickBird, 


- an image of retrieved surface reflectance. 


QUAC performs a fast and fairly accurate atmo- 
spheric correction if a) there are at least 10 diverse 
materials in a scene, and b) there are sufficiently 
dark pixels in a scene to allow for a good estimation 
of the baseline spectrum (Bernstein et al., 2008; Exe- 
lis QUAC, 2014; Exelis Atmospheric Correction, 
2014). Agrawal and Sarup (2011) compared the re- 
sults of applying both QUAC and FLAASH atmo- 
spheric corrections to EO-1 Hyperion hyperspectral 
data. 


« ATCOR—The ATmospheric CORrection program 
was originally developed at DLR, the German 
Aerospace Centre (Richter and Schlapfer, 2014). 
ATCOR consists of ATCOR 2 (used for flat terrain) 
and ATCOR 3 (used for rugged terrain, i.e., the 3 
stands for 3 dimensions). ATCOR 4 is used with 
remotely sensed data acquired by suborbital remote 
sensing systems. The atmospheric correction algo- 
rithm employs the MODTRAN 4+ (Alder-Golden 
et al., 1999, 2005) radiative transfer code to calcu- 
late look-up tables (LUT) of the atmospheric correc- 
tion functions (path radiance, atmospheric 
transmittance, direct and diffuse solar flux) that 
depend on scan angle, relative azimuth angle 
between scan line and solar azimuth, and terrain 
elevation. Examples of before and after atmospheric 
correction using ATCOR are shown in Figure 6-29. 
San and Suzen (2010), ERDAS ATCOR (2014) and 
Richter and Schlapfer (2014) provide detailed infor- 
mation about ATCOR. 


Artifact Suppression Most robust radiative transfer- 
based atmospheric correction algorithms also suppress 
artifacts in the corrected remote sensor data. For ex- 
ample, FLAASH uses an adjustable spectral “polish- 
ing” based on Alder-Golden et al. (1999). This type of 
spectral polishing involves applying mild adjustments 
to reflectance data so that the spectra appear more like 
spectra of real materials as recorded on the ground by 
a handheld spectroradiometer. ENVI’s EFFORT, Im- 
Spec’s ACORN, and Intergraph’s ATCOR all have ar- 
tifact suppression capability. 


Single Spectrum Enhancement Radiative transfer- 
based atmospheric correction programs (e.g., ACORN, 
FLAASH) allow the user to incorporate in situ spec- 
troradiometer measurements. This is commonly re- 
ferred to as single spectrum enhancement. The analyst 
simply provides an accurate in situ spectral reflectance 
curve for a known homogeneous area (e.g., a bare soil 
field of kaolinite, a deep water body, or a parking lot) 


and also provides the geometric coordinates of this 
same location in the remotely sensed data. The atmo- 
spheric correction program then determines all of the 
traditional radiative transfer scattering and absorption 
adjustments, plus it attempts to make the pixel in the 
image have the same spectral characteristics as the in 
situ spectroradiometric data. 


For example, an Analytical Spectral Devices (ASD) 
hand-held spectroradiometer (Figure 6-30a) was used 
to collect 46 in situ spectroradiometric measurements 
on the Savannah River Site Mixed Waste Management 
Facility (Figure 6-31a). GER DAIS 3715 hyperspectral 
data were collected and radiometrically corrected using 
ACORN and empirical line calibration (discussed in 
the next section). In addition, a single spectrum en- 
hancement was applied to both methods. A compari- 
son between the 46 in situ  spectroradiometer 
measurements for each of the 35 bands with the four 
atmospheric correction treatments (ELC, ACORN, 
ELC+SSE, ACORN+ SSE) reveals that the ACORN 
and ELC with the single spectrum enhancement pro- 
vided very accurate radiometric correction of the re- 
mote sensor data (Figure 6-3lb). The DAIS 3715 
sensor did not collect data in one of the optimum at- 
mospheric absorption windows. This may be the rea- 
son why ACORN did not do as well as a stand-alone 
correction. 


Absolute Atmospheric Correction Using 
Empirical Line Calibration 


Absolute atmospheric correction may also be per- 
formed using empirical line calibration (ELC), which 
forces the remote sensing image data to match in situ 
spectral reflectance measurements, hopefully obtained 
at approximately the same time and on the same date 
as the remote sensing overflight. Empirical line calibra- 
tion is based on the equation (Smith and Milton, 1999; 
ImSpec, 2014): 


BV, = p,A,+ By, (6.35) 


where BV; is the digital output value for a pixel in 
band k, p, equals the scaled surface reflectance of the 
materials within the remote sensor IFOV at a specific 
wavelength (A), A; is a multiplicative term affecting 
the BV, and B; is an additive term. The multiplicative 
term is associated primarily with atmospheric trans- 
mittance and instrumental factors, and the additive 
term deals primarily with atmospheric path radiance 
and instrumental offset (i.e., dark current). 


To use Empirical Line Calibration, the image analyst 
usually selects two or more areas in the scene with dif- 
ferent albedos (e.g., one bright target such as sand and 
one dark target such as a deep, nonturbid water body). 
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ATCOR® Atmospheric Correction 


c. Before atmospheric correction. 


d. After atmospheric correction. 


FIGURE 6-29 a) Natural color image containing substantial haze prior to atmospheric correction (Collwitz, Germany). b) Im- 
age after atmospheric correction using ATCOR. c) Color-infrared color composite containing substantial haze Image after 
atmospheric correction using ATCOR (images courtesy of Geosystems GmbH). 


The areas selected should be as homogeneous as possi- 
ble. Jn situ spectroradiometer measurements of these 
targets are made on the ground. The im situ and remote 
sensing—derived spectra are regressed and gain and off- 
set values computed. The gain and offset values are 
then applied to the remote sensor data on a band by 
band basis, removing atmospheric attenuation. Note 
that the correction is applied band by band and not 
pixel by pixel, as it is with ACORN, FLAASH, and 
ATCOR. 


To gain an appreciation of how empirical line calibra- 
tion works, consider how GER DAIS 3715 radiance 
values of the Savannah River Site were converted to 
scaled surface reflectance values. Instead of trying to 
locate natural dark and light radiometric control 
points on the ground, dark and light 8 X 8 m calibra- 
tion panel targets were located in the field at the time 
of a GER DAIS 3715 hyperspectral overflight (Figure 
6-30b). Moran et al. (2001) provide guidance on how 
to prepare and maintain reference reflectance tarps. 
McCoy (2005) and Johannsen and Daughtry (2009) 
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a. In situ spectroradiometer calibration. 


b. Calibration targets. 


FIGURE 6-30 a) Field crew taking a spectroradiometer measurement from a calibrated reflectance standard on the tripod. b) 
8% 8 m black-and-white calibration targets at the Savannah River Site (Jensen et al., 2003). 


provide additional information about how to collect in 
situ spectral reflectance data. 


The spatial resolution of the imagery was 2.4 XM 2.4 m. 
Image pixel brightness values for the targets were ex- 
tracted from flightline 05 at two locations: dark 
(3,682,609 N; 438,864 E) and bright (3,682,608 N; 
438,855 E). In situ ASD spectroradiometer measure- 
ments were obtained from the calibration targets. The 
remote sensing and in situ spectral measurements for 
the calibration targets were then paired and input to 
the empirical line calibration to derive the appropriate 
gain and offset values to atmospherically correct the 
hyperspectral data. Results of the empirical line cali- 
bration are summarized in Figure 6-31b. 


It is important to note that most multispectral remote 
sensing datasets can be calibrated using empirical line 
calibration. The difficulty arises when trying to locate 
homogeneous bright and dark targets in the study, col- 
lecting representative in situ spectroradiometer mea- 
surements, and extracting uncontaminated pixels of 
the calibration targets from the imagery. If the analyst 
does not have access to in situ spectra obtained at the 
time of the remote sensing overflight, it might be possi- 
ble to use spectra of such fundamental materials as 
clear water and sand (quartz) that are stored in spec- 
tral libraries. 


Laboratory-derived spectra may be found at the AS- 
TER Spectral Library (2014), which contains the: 
Johns Hopkins Spectral Library, the NASA Jet Pro- 
pulsion Laboratory Spectral Library, and the US. 
Geological Survey spectral library (Baldridge et al., 
2009). Hopefully, some of these materials exist in the 
scene and the analyst can locate the appropriate pixel 
and pair the image brightness values with the library in 
situ spectroradiometer data. 


To demonstrate the use of empirical line calibration 
applied to historical imagery, consider the Landsat 
Thematic Mapper dataset shown in Figure 6-32a. In 
situ spectral reflectance measurements were not ob- 
tained when the Landsat TM imagery were acquired 
on February 3, 1994. Therefore, how can the bright- 
ness values in this historical imagery be converted into 
scaled surface reflectance? The answer lies in the use of 
empirical line calibration, in situ library spectra, and 
the Landsat TM brightness values (BV; ; ,). This par- 
ticular scene has a substantial amount of water and 
beach. Therefore, in situ library spectroradiometer data 
of water (from the Johns Hopkins library) and quartz 
(from the NASA JPL library) were utilized. The multi- 
spectral brightness values of one pixel of water and one 
pixel of beach were extracted from the Landsat TM da- 
ta. These were paired with the in situ spectral reflec- 
tance measurements in the same six wavelengths (the 
Landsat 6 thermal infrared channel was not used). The 
empirical line calibration resulted in a Landsat TM da- 
taset scaled to scaled surface reflectance. For example, 
consider a pixel of healthy loblolly pine. The original 
brightness value response is shown in Figure 6-32b. 
The scaled surface reflectance derived from the Land- 
sat TM data is shown in Figure 6-32c. All pixels in the 
scene now represent scaled surface reflectance and not 
brightness value. 


Scaled surface reflectance measurements obtained 
from this date of imagery can now be compared with 
scaled surface reflectance measurements extracted 
from other dates of Landsat TM imagery for purposes 
of monitoring biomass, etc. Of course, the use of li- 
brary spectra is not as good as acquiring in situ spec- 
troradiometry at selected sites in the scene at the time 
of the remote sensing data collection. Nevertheless, the 
technique is useful. 
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Relationship between 46 Jn Situ Spectroradiometer Measurements and Pixel Spectral 
Reflectance after Processing Using Empirical Line Calibration and ACORN 
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FIGURE 6-31 a) Forty-six in situ ASD p € ro ad ome e (400 - 2,500 nm) measurements were obtained on the Savannah 
River Site Mixed Waste Management Facility near Aiken, SC. b) The relationship between the 46 in situ spectroradiometer 
measurements and pixel spectral reflectance after processing using empirical line calibration (ELC) and ACORN. The ELC and 
ACORN with the single spectrum enhancements provided the most accurate atmospheric correction of the GER DAIS 3715 


hyperspectral data (based on Jensen et al., 2003). 


Relative Radiometric Correction of 
Atmospheric Attenuation 


As previously discussed, absolute radiometric correc- 
tion makes it possible to relate the digital counts in sat- 
ellite or aircraft image data to scaled surface 
reflectance. Except when using empirical line calibra- 
tion, this requires sensor calibration coefficients (to 


convert the original remote sensor data to W m? sr!) 
and an atmospheric correction algorithm usually based 
on radiative transfer code. Unfortunately, the applica- 
tion of these codes to a specific scene and date also re- 
quires knowledge of both the sensor spectral profile 
and the atmospheric properties at the time of remote 
sensor data collection (Du et al., 2002). If all of this 
type of information is available, then it is possible for 
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the atmospheric radiative transfer code to provide real- 
istic estimates of the effects of atmospheric scattering 
and absorption and convert the imagery to scaled sur- 
face reflectance. If such information is not available, 
then relative radiometric correction techniques may be 
used. 


Relative radiometric correction may be used 1) to nor- 
malize the intensities among the different bands within 
a single-date remotely sensed image, and 2) to normal- 
ize the intensities of bands of remote sensor data in 
multiple dates of imagery to a standard scene selected 
by the analyst. 


Single-Image Normalization Using Histogram 
Adjustment 


This simple method is based primarily on the fact that 
infrared data (>0.7 Xm) are largely free of atmospheric 
scattering effects, whereas the visible region (0.4 — 0.7 
Xm) is strongly influenced by them. The method in- 
volves evaluating the histograms of the various bands 
of remotely sensed data of the desired scene. Normally, 
the data collected in the visible wavelengths (e.g., TM 
bands | to 3) have a higher minimum value because of 
the increased atmospheric scattering taking place in 
these wavelengths. For example, a histogram of 
Charleston, SC, TM data (Figure 6-33) reveals that it 
has a minimum of 51 and a maximum of 242. Con- 
versely, atmospheric absorption subtracts brightness 
from the data recorded in the longer-wavelength inter- 
vals (e.g., TM bands 4, 5, and 7). This effect commonly 
causes data from the infrared bands to have minimums 
close to zero, even when few objects in the scene truly 
have a reflectance of zero (Figure 6-33). 


If the histograms in Figure 6-33 are shifted to the left 
so that zero values appear in the data, the effects of at- 
mospheric scattering will be somewhat minimized. 
This simple algorithm models the first-order effects of 
atmospheric scattering, or haze. It is based on a sub- 
tractive bias established for each spectral band. The 
bias may also be determined by evaluating a histogram 
of brightness values of a reference target such as deep 
water in all bands. The atmospheric effects correction 
algorithm is defined: 

output BV; ;, = input BV; ;;— bias (6.36) 
where input BV; ; , = input pixel value at line 7 and col- 
umn j of band k and output BV; ;, = the adjusted pixel 
value at the same location. 


In this example, the appropriate bias was determined 
for each histogram in Figure 6-33 and subtracted from 
the data. The histograms of the adjusted data are dis- 
played in Figure 6-34. It was not necessary to adjust 


bands 5 and 7 for first-order atmospheric effects be- 
cause they originally had minimums of zero. Hadjimi- 
tisis et al. (2010) used histogram adjustment (.e., 
darkest pixel) atmospheric correction and found it of 
value for analyzing Landsat TM data for agricultural 
applications. Chrysoulakis et al. (2010) also found the 
simple histogram adjustment method to be useful. 


Multiple-Date Image Normalization Using 
Regression 


Multiple-date image normalization involves selecting a 
base image (6) and then transforming the spectral 
characteristics of all other images obtained on different 
dates (e.g., those obtained on dates b-1, b-—2 and/or 
b+1, b+2, etc.) to have approximately the same radio- 
metric scale as the base image. It is important to re- 
member, however, that the radiometric scale used in a 
relative multiple-date image normalization will most 
likely be simple brightness values (e.g., BV with a range 
from 0 to 255) rather than scaled surface reflectance 
produced when conducting an absolute radiometric 
correction. 


Multiple-date image normalization involves the selec- 
tion of pseudo-invariant features (PIFs), often referred 
to as radiometric ground control points. To be of value 
in the multiple-date image normalization process, the 
pseudo-invariant features should have the following 
characteristics: 


¢ The spectral characteristics of a PIF should change 
very little through time, although it is acknowledged 
that some change is inevitable. Deep nonturbid 
water bodies, bare soil, large rooftops, or other 
homogeneous features are candidates. 


¢ The PIF should be at approximately the same eleva- 
tion as the other land in the scene. Selecting a 
mountaintop PIF would be of little use in estimat- 
ing atmospheric conditions near sea level because 
most aerosols in the atmosphere occur within the 
lowest 1,000 m. 


¢ The PIF should normally contain only minimal 
amounts of vegetation. Vegetation spectral reflec- 
tance can change over time as a result of environ- 
mental stress and plant phenology. However, an 
extremely stable, homogeneous forest canopy 
imaged on approximate near-anniversary dates 
might be considered. 


¢ The PIF must be in a relatively flat area so that 
incremental changes in Sun angle from date to date 
will have the same proportional increase or decrease 
in direct beam sunlight for all normalization targets. 


Regression is used to relate the base image’s PIF spec- 
tral characteristics with PIF spectra characteristics 
from other dates of imagery. The algorithm assumes 
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FIGURE 6-32 a) A Landsat Thematic Mapper image acquired on February 3, 1994, was radiometrically corrected using em- 
pirical line calibration and paired NASA JPL and Johns Hopkins University spectral library beach and water in situ spectroradi- 
ometer measurements and Landsat TM image brightness values (BV;  ,). Original imagery courtesy of NASA. b) A pixel of 
loblolly pine with its original brightness values in six bands (the TM band 6 thermal channel was not used). c) The same pixel 
after empirical line calibration to scaled surface reflectance. Note the correct chlorophyll absorption in the blue (band 1) and 
red (band 3) portions of the spectrum and the increase in near-infrared reflectance. 
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FIGURE 6-33 Original histograms of six bands of the Charleston, SC, Thematic Mapper scene. Atmospheric scattering in the 
visible regions has increased the minimum brightness values in bands 1, 2, and 3. Generally, the shorter the wavelengths 
sensed by each band, the greater the offset from a brightness value of zero. 


that the pixels sampled at time b + 1 orb—1arelinear- | Numerous scientists have investigated the utility of 
ly related to the pixels for the same locations on the _ pseudo-invariant features to normalize multiple-date 
base image (4). This implies that the spectral reflec- imagery. For example, Caselles and Garcia (1989), 
tance properties of the sampled pixels have not Schott et al. (1988), and Hall et al. (1991) developed a 
changed during the time interval. Thus, the key to the — radiometric rectification technique that corrected imag- 
image regression method is the selection of quality es of the same areas through the use of landscape ele- 
pseudo-invariant features. ments whose reflectances were nearly constant over 

time. Jensen et al. (1995) used similar procedures to 
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FIGURE 6-34 Results of applying a simple histogram adjustment atmospheric scattering correction to the data shown in Fig- 
ure 6-33. Only the first four Thematic Mapper bands required the adjustment. This method does not correct for atmospheric 


absorption. 


perform relative radiometric correction. Heo and 
Fitzhugh (2000) developed different techniques to se- 
lect PIFs. Du et al. (2002) developed a procedure for 
the relative radiometric normalization of multitempo- 
ral satellite images where PIFs were selected objectively 
from images that were analyzed using principal com- 
ponent analysis (PCA). 


A remote sensing study that identified changes in cat- 
tail distribution in the South Florida Water Manage- 
ment District can be used to demonstrate the 
radiometric normalization process (Jensen et al., 1995). 
Five predominantly cloud-free dates of satellite remote 
sensor data were collected by two sensor systems for 
Everglades Water Conservation Area 2A from 1973 to 
1991. Landsat Multispectral Scanner (MSS) data were 
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TABLE 6-6 Characteristics of the remotely sensed satellite data used to inventory wetland in Water Conservation Area 2A of 
the South Florida Water Management District (Jensen et al., 1995). 


Date Type of Imagery Bands Used Nominal Instantaneous Rectification RMSE 
Field of View (m) 
3/22/73 Landsat MSS 1,2,4 79K 79 -0.377 
4/02/76 Landsat MSS |, 2,4 79 79 =0.27/5 
10/17/82 Landsat MSS 1,2,4 79 79 -0.807 
4/04/87 SPOT HRV 1,23 20M 20 -0.675 
8/10/91 SPOT HRV 1,23 20K 20 -0.400 


obtained in 1973, 1976, and 1982 and SPOT High Res- 
olution Visible (HRV) multispectral (XS) data in 1987 
and 1991. The specific date, type of imagery, bands 
used in the analysis, and nominal spatial resolution of 
the sensor systems are summarized in Table 6-6. Twen- 
ty (20) ground control points (GCPs) were obtained 
and used to rectify the August 10, 1991, remote sensor 
data to a standard map projection. The remotely 
sensed data were rectified to a Universal Transverse 
Mercator (UTM) map projection having 20 X 20 m 
pixels using a nearest-neighbor resampling algorithm 
and a root-mean-square error (RMSE) of —0.4 pixel 
(Rutchey and Vilchek, 1994). All other images were 
resampled to 20 M 20 m pixels using nearest-neighbor 
resampling and registered to the 1991 SPOT data for 
change detection purposes. The RMSE statistic for 
each image is summarized in Table 6-6. 


A problem associated with using historical remotely 
sensed data for change detection is that the data are 
usually from non-anniversary dates with varying Sun 
angle, atmospheric, and soil moisture conditions. The 
multiple dates of remotely sensed data should be nor- 
malized so that these effects can be minimized or elimi- 
nated (Hall et al., 1991). 


Differences in direct-beam solar radiation due to varia- 
tion in Sun angle and Earth-to-Sun distance can be 
calculated accurately, as can variation in pixel BVs due 
to detector calibration differences between sensor sys- 
tems. However, removal of atmospheric and phase an- 
gle effects requires information about the gaseous and 
aerosol composition of the atmosphere and the bidi- 
rectional reflectance characteristics of elements within 
the scene (Eckhardt et al., 1990). Because atmospheric 
and bidirectional reflectance information were not 
available for any of the five scenes, an “empirical scene 
normalization” approach was used to match the detec- 
tor calibration, astronomic, atmospheric, and phase 
angle conditions present in a reference scene. The Au- 
gust 10, 1991, SPOT HRV scene was selected as the 


base scene to which the 1973, 1976, 1982, and 1987 
scenes were normalized. The 1991 SPOT image was se- 
lected because it was the only year for which quality in 
situ ground reference data were available. 


Image normalization was achieved by applying regres- 
sion equations to the 1973, 1976, 1982, and 1987 imag- 
ery to predict what a given BV would be if it had been 
acquired under the same conditions as the 1991 refer- 
ence scene. These regression equations were developed 
by correlating the brightness of pseudo-invariant fea- 
tures present in both the scene being normalized and 
the reference (1991) scene. PIF were assumed to be 
constant reflectors, so any changes in their brightness 
values were attributed to detector calibration, and as- 
tronomic, atmospheric, and phase angle differences. 
Once these variations were removed, changes in BV 
could be related to changes in surface conditions. 


Multiple wet (water) and dry (eg., unvegetated bare 
soil) PIF were found in the base year image (1991) and 
each of the other dates of imagery (e.g., 1987 SPOT da- 
ta). A total of 21 radiometric control points were used 
to normalize the 1973, 1976, 1982, and 1987 data to 
the 1991 SPOT data. It is useful to summarize the na- 
ture of the normalization targets used and identify ad- 
justments that had to be made when trying to identify 
dry soil targets in a humid subtropical environment. 
Radiometric normalization targets found within the 
1987 and 1991 SPOT data consisted of three wet points 
obtained just to the north of WCA-2A within WCA-1 
and three dry points extracted from an excavated area, 
a dry lake area, and a limestone road area. The bright- 
ness values of the early image targets (e.g., 1987) were 
regressed against the brightness values of the base im- 
age targets (e.g., 1991) for each band (Figure 6-35a—c). 
The coefficients and intercept of the equation were 
used to compute a normalized 1987 SPOT dataset, 
which had approximately the same spectral character- 
istics as the 1991 SPOT data. Each regression model 
contained an additive component that corrected for 
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Radiometric Normalization of Multiple-date South Florida SPOT Data 
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the difference in atmospheric path radiance among 
dates and a multiplicative term that corrected for the 
difference in detector calibration, Sun angle, Earth— 
Sun distance, atmospheric attenuation, and phase an- 
gle between dates. 


The 1982 MSS data were normalized to the 1991 data 
using (1) three common wet targets found within 
WCA-1 and (2) two dry points extracted from a bare 
soil excavation area in 1982, which progressed north- 
ward about 300 m (15 pixels) in the y dimension by 
1991 (1.e., the x dimension was held constant). Thus, 
two noncommon dry radiometric control points were 
extracted for this date. Hall et al. (1991) suggest that 
the members of the radiometric control sets may not be 
the same pixels from image to image, in contrast to 
geometric control points for spatial image rectification, 
which are composed of identical elements in each 
scene. Furthermore, they suggest that “using fixed ele- 
ments inevitably requires manual selection of sufficient 
numbers of image-to-image pairs of suitable pixels, 
which can be prohibitively labor intensive, particularly 
when several images from a number of years are being 
considered.” Such conditions were a factor in the Ever- 
glades study. 


The 1976 MSS data were normalized to the 1991 data 
using three wet targets located in WCA-1 and two dry 
points extracted along a bare soil road and a limestone 
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FIGURE 6-35 a) Relationship between the same wet and 
dry regions found in both the 4/4/87 and 8/10/91 SPOT 
band 1 (green) dataset. The equation was used to normalize 
the 4/4/87 data to the 8/10/91 SPOT data as per methods 
described in Eckhardt et al. (1990). b) Relationship between 
wet and dry regions found in both the 4/4/87 and 8/10/91 
SPOT band 2 (red) dataset. c) Relationship between wet and 
dry regions found in both the 4/4/87 and 8/10/91 SPOT 
band 3 (near-infrared) dataset (Jensen, J. R., Rutchey, K., 
Koch, M. and S. Narumalani,1995, “Inland Wetland Change 
Detection in the Everglades Water Conservation Area 2A 
Using a Time Series of Normalized Remotely Sensed Data,” 
Photogrammetric Engineering & Remote Sensing, 
61(2):199-209. Reprinted with permission of the American 
Society for Photogrammetry & Remote Sensing). 


100 120 140 


bare soil area. The 1973 MSS data were normalized to 
the 1991 data using two wet and three dry targets. The 
greater the time between the base image (e.g., 1991) 
and the earlier year’s image (e.g., 1973), the more diffi- 
cult it is to locate unvegetated, dry normalization tar- 
gets. For this reason, analysts sometimes use synthetic, 
pseudo-invariant features such as concrete, asphalt, 
rooftops, parking lots, and roads when normalizing 
historic remotely sensed data (Schott et al., 1988; Ca- 
selles and Garcia, 1989; Hall et al., 1991). 


The normalization equations for each date are summa- 
rized in Table 6-7. The gain (slope) associated with the 
SPOT data was minimal, while the historical MSS data 
required significant gain and bias adjustments (be- 
cause some MSS data were not originally acquired as 
8-bit data). The methodology applied to all images 
minimized the differences in Sun angle, atmospheric ef- 
fects, and soil moisture conditions between the dates. 
The radiometrically corrected remote sensor data were 
then classified and used to monitor wetland change 
(Jensen et al., 1995). 


The ability to use remotely sensed data to classify land 
cover accurately is contingent on there being a robust 
relationship between remotely sensing brightness value 
(BV) and actual surface conditions. However, factors 
such as Sun angle, Earth-Sun distance, detector cali- 
bration differences among the various sensor systems, 
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TABLE 6-7 Equations used to normalize the radiometric 
characteristics of the historic remote sensor data with the 
August 10, 1991, SPOT XS data (Jensen et al., 1995). 


Date Band Slope y-intercept ia 
3/22/73 MSS 1 1.40 SAY O1098 
2 1.01 23.49 0.98 
4 3.28 23.48 0.99 
4/02/76 MSS 1 0.57 31.69 O99. 
2 0.43 21.91 0.98 
4 3.84 26.32 0.96 
10/17/82 MSS 1 2ao2, Nel 0.99 
2 2.142 8.488 0.99 
4 nAZ) WAg36 0.99 
4/04/87 SPOT 1 1.025 21.152 0.99 
2 0.987 9.448 0.98 
3 1.045 13.263 0.95 
All regression equations were significant at the 0.001 level. 


atmospheric condition, and Sun—target—sensor (phase 
angle) geometry will affect pixel brightness value. Im- 
age normalization reduces pixel BV variation caused 
by nonsurface factors, so that variations in pixel 
brightness value among dates may be related to actual 
changes in surface conditions. Normalization may al- 
low the use of pixel classification logic developed from 
a base year scene to be applied to the other normalized 
scenes. 


Correcting for Slope and 
Aspect Effects 


The previous sections discussed how scattering and ab- 
sorption in the atmosphere can attenuate the radiant 
flux recorded by the sensor system. Topographic slope 
and aspect may also introduce radiometric distortion 
of the recorded signal. In some locations, the area of 
interest might even be in complete shadow, dramatical- 
ly affecting the brightness values of the pixels involved. 
For these reasons, research has been directed toward 
the removal of topographic effects, especially in moun- 


tainous regions, on Landsat and SPOT digital multi- 
spectral data (Shepherd et al., 2014). The goal of a 
slope-aspect correction is to remove topographically 
induced illumination variation so that two objects hav- 
ing the same reflectance properties show the same 
brightness value in the image despite their different ori- 
entation to the Sun’s position. If the topographic 
slope-aspect correction is applied effectively, the three- 
dimensional impression we get when looking at a satel- 
lite image of mountainous terrain should be somewhat 
subdued. A good slope-aspect correction is believed to 
improve forest stand classification when compared to 
noncorrected imagery (Civco, 1989; Meyer et al., 
1993). 


Teillet et al. (1982) described four topographic slope- 
aspect correction methods: the simple cosine correc- 
tion, two semi-empirical methods (the Minnaert meth- 
od and the C correction), and a statistic-empirical 
correction. Each correction is based on i//umination, 
which is defined as the cosine of the incident solar an- 
gle, thus representing the proportion of the direct solar 
radiation hitting a pixel. The amount of illumination is 
dependent on the relative orientation of the pixel to- 
ward the Sun’s actual position (Figure 6-36). Each 
slope-aspect topographic correction method to be dis- 
cussed requires a digital elevation model (DEM) of the 
study area. The DEM and satellite remote sensor data 
(e.g., Landsat TM data) must be geometrically regis- 
tered and resampled to the same spatial resolution 
(e.g., 30% 30 m pixels). The DEM is processed so that 
each pixel’s brightness value represents the amount of 
illumination it should receive from the Sun. This infor- 
mation is then modeled using one of the four algo- 
rithms to enhance or subdue the original brightness 
values of the remote sensor data. 


The Cosine Correction 


The amount of irradiance reaching a pixel on a slope is 
directly proportional to the cosine of the incidence an- 
gle i, which is defined as the angle between the normal 
on the pixel in question and the zenith direction (Teil- 
let et al., 1982). This assumes 1) Lambertian surfaces, 
2) a constant distance between Earth and the Sun, and 
3) a constant amount of solar energy illuminating 
Earth (somewhat unrealistic assumptions!). Only the 
part cos 7 of the total incoming irradiance, E,, reaches 
the inclined pixel. It is possible to perform a simple 
topographic slope-aspect correction of the remote sen- 
sor data using the following cosine equation: 


cos o. 


Ly z=L 
H T cos i 


(6.37) 


where 
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FIGURE 6-36 Representation of the Sun’s angle of inci- 
dence, i, and the solar zenith angle, O.. 


Ly=radiance observed fora horizontal surface (i.e., 
slope-aspect-corrected remote sensor data) 


Lr= radiance observed over sloped terrain (i.e., the 
raw remote sensor data) 


9, = Sun’s zenith angle 


i=Sun’s incidence angle in relation to the normal 
on a pixel (Figure 6-36). 


Unfortunately, this method models only the direct part 
of the irradiance that illuminates a pixel on the ground. 
It does not take into account diffuse skylight or light 
reflected from surrounding mountainsides that may il- 
luminate the pixel in question. Consequently, weakly 
illuminated areas in the terrain receive a disproportion- 
ate brightening effect when the cosine correction is ap- 
plied. Basically, the smaller the cos i, the greater the 
overcorrection is (Meyer et al., 1993). Nevertheless, 
several researchers have found the cosine correction of 
value. For example, Civco (1989) achieved good results 
with the technique when used in conjunction with em- 
pirically derived correction coefficients for each band 
of imagery. 


The Minnaert Correction 


Teillet et al. (1982) introduced the Minnaert correction 
to the basic cosine function: 


cos 8k 
Ly = L{—) 
H TX cos i 


where k = the Minnaert constant. 


(6.38) 


The constant varies between 0 and 1| and is a measure 
of the extent to which a surface is Lambertian. A per- 
fectly Lambertian surface has k = 1 and represents a 
traditional cosine correction. Meyer et al. (1993) de- 
scribe how k may be computed empirically. Lu et al. 
(2008) developed a pixel-based Minnaert correction 
method which reduced topographic effects when ap- 
plied to Landsat ETM™ data over mountainous ter- 
rain. 


A Statistical-Empirical Correction 


For each pixel in the scene, it is possible to correlate (1) 
the predicted illumination (cos i % 100) from the DEM 
with (2) the actual remote sensor data. For example, 
Meyer et al. (1993) correlated Landsat TM data of 
known forest stands in Switzerland with the predicted 
illumination from a high-resolution DEM. Any slope 
in the regression line suggests that a constant type of 
forest stand will appear differently on different terrain 
slopes. Conversely, by taking into account the statisti- 
cal relationship in the distribution, the regression line 
may be rotated based on the following equation: 


Ly = Le- cos(i)m—b+L> (6.39) 


where 


Ly = radiance observed for a horizontal surface 
(i.e., slope-aspect-corrected remote sensor 
data) 

Ly= radiance observed over sloped terrain (1.e., 
the raw remote sensor data) 


Lig = average of L for forested pixels (according 
to ground reference data) 


i= Sun’s incidence angle in relation to the nor- 
mal on a pixel (Figure 6-36) 


m = slope of the regression line 


b = y-intercept of the regression line. 


Application of this equation makes a specific object 
(e.g., a particular type of deciduous forest) indepen- 
dent of cos i and produces the same brightness values 
(or radiance) throughout the image for this object. 


The C Correction 


Teillet et al. (1982) introduced an additional adjust- 
ment to the cosine function called the c correction: 


cos O,te 


é (6.40) 


ey je 
H T cos ite 
where 
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b. ; : . 
c= — in the previous regression equation. 
m 


Similar to the Minnaert constant, c increases the de- 
nominator and weakens the overcorrection of faintly 
illuminated pixels. 


Local Correlation Filter 


Shepherd et al. (2014) used a local correlation filter ap- 
plied to a panchromatic band of imagery and a digital 
elevation model of the same area. The 7 & 7 spatial 
moving filter determined the offset between a digital el- 
evation model (DEM) and ortho-rectified satellite im- 
agery for every pixel. The mean correlation coefficient 
between the input panchromatic band and the DEM 
within the moving 7K 7 pixel window was increased to 
0.71 by using the local correlation filter. They suggest- 
ed their correction removed the majority of the high- 
frequency artifacts. 
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7 GEOMETRIC CORRECTION 


It would be wonderful if every remotely sensed image 
contained data that were already in their proper geo- 
metric x, y locations. This would allow each image to 
be used as if it were a map. Unfortunately, this is not 
the case. Instead, it is usually necessary to preprocess 
the remotely sensed data and remove the geometric dis- 
tortion so that individual picture elements are in their 
proper planimetric (x, y) map locations (Purkis and 
Klemas, 2011). This allows remote sensing—derived in- 
formation to be related to other thematic data in geo- 
graphic information systems (GIS) (Merchant and 
Narumalani, 2009; Jensen and Jensen, 2013) or spatial 
decision support systems (SDSS) (Marcomini et al., 
2008; Sugumaran and DeGrotte, 2010). Geometrically 
corrected imagery can be used to extract accurate dis- 
tance, polygon area, and direction (bearing) informa- 
tion (Gonzalez and Woods, 2007; Wolf et al., 2013). 
Geometrically inaccurate remote sensor data can have 
very serious impacts on remote sensing—derived prod- 
ucts (McRoberts, 2010). 


Internal and External 
Geometric Error 


Remotely sensed imagery typically exhibits internal 
and external geometric error. It is important to recog- 
nize the source of the internal and external error and 
whether it is systematic (predictable) or nonsystematic 
(random). Systematic geometric error is generally easi- 
er to identify and correct than random geometric error. 


Internal Geometric Error 


Internal geometric errors are generally introduced by 
the remote sensing system itself or in combination with 


Earth rotation or curvature characteristics. These dis- 
tortions are often systematic (predictable) and may be 
identified and then corrected using prelaunch or in- 
flight platform ephemeris (i.e., information about the 
geometric characteristics of the sensor system and the 
Earth at the time of data acquisition). Geometric dis- 
tortions in imagery that can sometimes be corrected 
through analysis of sensor characteristics and ephem- 
eris data include: 


* skew caused by Earth rotation effects, 


* scanning system-—induced variation in nominal 
ground resolution cell size, 


* scanning system one-dimensional relief displace- 
ment, and 


* scanning system tangential scale distortion. 


Image Offset (Skew) Caused by Earth Rotation 
Effects 


Earth-observing Sun-synchronous satellites are nor- 
mally in fixed orbits that collect a path (or swath) of 
imagery as the satellite makes its way from the north to 
the south in descending mode (Figure 7-la). Mean- 
while, the Earth below rotates on its axis from west to 
east making one complete revolution every 24 hours. 
This interaction between the fixed orbital path of the 
remote sensing system and the Earth’s rotation on its 
axis skews the geometry of the imagery collected. For 
example, consider just three hypothetical scans of 16 
lines each obtained by the Landsat Enhanced Themat- 
ic Mapper Plus (ETM”). If the data are not deskewed, 
they will appear in the dataset incorrectly, as shown in 
Figure 7-1b. While this looks correct, it is not. This 
matrix does not take into account the Earth rotation 
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effects. The data in this dataset are actually skewed to 
the east by a predictable amount. 


Conversely, if the remotely sensed data are deskewed, 
then all of the pixels associated with a single scan (con- 
taining 16 lines of data) will be offset (adjusted) by the 
digital image processing system a certain amount to 
the west (Figure 7-1c). Deskewing is defined as the sys- 
tematic displacement of pixels westward in a frame of 
imagery to correct for the interaction of the satellite 
sensor system’s angular velocity and the Earth’s sur- 
face velocity. This adjustment places all of the pixels in 
the scan in their proper positions relative to adjacent 
scans. Null values are added to the dataset to maintain 
the integrity of the raster (matrix) format. The amount 
of shift or displacement to the west is a function of the 
relative velocities of both the satellite and the Earth 
and the length of the image frame that is recorded. 


Most satellite image data providers automatically 
deskew the data that is delivered to customers using 
the logic shown in Figure 7-1c. First, the surface veloc- 
ity of the Earth, v is computed, 


earth»? 


v = Dearth 1 COSA (7.1) 


earth 


where r is the radius of the Earth (6.37816 Mm) and 
ann i8 the Earth’s rotational velocity (72.72 Mrad s"!) 
at a specific latitude, 1. Therefore, for Charleston, SC, 
located at 33° N latitude, the surface velocity of the 
Earth is: 


= 72.72 prad s‘ x 6.37816 Mm x 0.83867 


Vearth 


Vea = 389 ms. 

Next we must determine the length of time it takes for 
the satellite to scan a typical remote sensing frame (F) 
of data on the ground. In this example we will consider 
a 185-km frame obtained by Landsats 1, 2, and 3 asso- 
ciated with the Multispectral Scanner and Landsats 4, 
5, and 7 associated primarily with the Thematic Map- 
per and Enhanced Thematic Mapper Plus. Landsats 1, 
2, and 3 have an angular velocity, @),, 123, of 1.014 
mrad s7!, Landsat satellites 4,5, and 7 have an angular 
velocity, @janq4s7, Of 1.059 mrad s! (Williams, 2003).! 
Therefore, a typical 185-km frame of Landsat MSS im- 
agery (WRSI orbit) would be scanned in: 


ae (7.2) 


t 
rX Mjand123 


1. For Landsats 1,2, and 3(WRS1 orbit) the angular velocity is (251 
paths/cycle 27 1,000 mrad/path) / (18 days/cycleX 86,400 s/day) 
= 1.014 mrad/s. For Landsats 4, 5, and 7 (WRS2 orbit) the angular 
velocity is (233 paths/cycle ¥ 2 2 XH 1,000 mrad/path) / (16 days/cycle 
& 86,400 s/day) = 1.059 mrad/s. 


185 km 


Sn eee a OE ge 
(6.37816 Mm)(1.014 mrad s ') 


qf 


A 185-km frame of Landsat 4, 5 and 7 imagery (WRS2 
orbit) would be scanned in: 


185 km 


= OO iF 
(6.37816 Mm)(1.059 mrad s') 


a, 


Therefore, during the time that the 185-km frame of 
Landsat 1, 2, or MSS imagery was collected at 
Charleston, SC (32° N latitude), the surface of the 
Earth moved to the east by: 

Ax =v 


earth * St Us 3) 


east 


AXeast = 389 ms’ x 28.6s = 11.12 km. 
Similarly, during the time a 185-km frame of Landsat 
ETM* imagery is collected, the surface of the Earth 
moves to the east by: 


AX east = 389 ms x 27.39 s= 10.65 km. 

This is approximately 6% of the 185-km frame size 
(e.g., 11.12/185 = 0.06; 10.65/185 = 0.057) for Landsats 
1, 2, and 3 MSS and Landsat Thematic Mappers 4, 5, 
and 7 data. Fortunately, much of the satellite imagery 
provided by commercial and public remote sensing 
data providers have already been deskewed by the ap- 
propriate amount per line scan. 


The skewing described holds true for line-scan sensors 
such as Landsat TM and ETM” and for pushbroom 
sensors such as SPOT HRV, DigitalGlobe’s World- 
View-2, and GeoEye’s GeoEye-1. Every fixed-orbit re- 
mote sensing system that collects data while the Earth 
rotates on its axis will contain frames of imagery that 
are skewed. 


Scanning System-Induced Variation in Ground 
Resolution Cell Size 


A large amount of remote sensor data is acquired us- 
ing scanning systems (e.g., Landsat 7, ASTER). Fortu- 
nately, an orbital multispectral scanning system scans 
through just a few degrees off-nadir as it collects data 
hundreds of kilometers above the Earth’s surface (e.g., 
Landsat 7 data are collected at 705 km AGL). This 
configuration minimizes the amount of distortion in- 
troduced by the scanning system. Conversely, a subor- 
bital multispectral scanning system may be operating 
just tens of kilometers AGL with a scan field of view of 
perhaps 70 degrees. This introduces numerous types of 
geometric distortion that can be difficult to correct. 
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Correction of Image Offset (Skew) Caused by Earth Rotation Effects 


Equatorial 
plane and 
direction of 
Earth rotation:—> 
west to east 


Landsats 4, 5, 
7 at 9:45 a.m. 
local time 


i) 
i 


a. 
Pixels in a Landsat Thematic Pixels in a Landsat Thematic 
Mapper dataset prior to correcting Mapper dataset corrected 
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FIGURE 7-1 a) Landsat satellites 4, 5, and 7 are in a Sun-synchronous orbit with an angle of inclination of 98.2°. The Earth ro- 
tates on its axis from west to east as imagery is collected. b) Pixels in three hypothetical scans (consisting of 16 lines each) of 
Landsat TM data. While the matrix (raster) may look correct, it actually contains systematic geometric distortion caused by 
the angul ar velocity of the satellite in its descending orbital path in conjunction with the surface velocity of the Earth as it ro- 
tates on its axis while collecting a frame of imagery. c) The result of adjusting (deskewing) the original Landsat TM data to the 
west to compensate for Earth rotation effects. Landsats 4, 5, and 7 use a bidirectional cross-track scanning mirror (NASA, 
1998). 
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FIGURE 7-2 The ground resolution cell size along a single across-track scan is a function of a) the distance from the aircraft 
to the observation where H is the altitude of the aircraft above ground level (AGL) at nadir and H sec @ off-nadir; b) the in- 
stantaneous-field-of-view of the sensor, B , measured in radians; and c) the scan angle off-nadir, ¢ . Pixels off-nadir have 
semimajor and semiminor axes (diameters) that define the resolution cell size. The total field of view of one scan line is 0. 
One-dimensional relief displacement and tangential scale distortion occur in the direction perpendicular to the line of flight 


and parallel with a line scan. 


The ground swath width (gsw) is the length of the ter- 
rain strip remotely sensed by the system during one 
complete across-track sweep of the scanning mirror. It 
is a function of the total angular field of view of the 
sensor system, 9, and the altitude of the sensor system 
above ground level, H as shown in Figure 7-2. It is 
computed using the equation: 


gsw = tan(2) xHx2. (7.4) 


For example, the ground swath width of an across- 
track scanning system with a 90° total field of view and 
an altitude above ground level of 6,000 m would be 
12,000 m: 


gsw = tan( 22) x 6,000 x 2 


gsw = 1x 6,000 x 2 


gsw = 12,000 m. 


Most scientists using across-track scanner data use 
only approximately the central 70% of the swath width 
(35% on each side of nadir) primarily because ground 
resolution elements have larger cell sizes the farther 
they are away from nadir. 


The diameter of the circular ground area viewed by the 
sensor, D, at nadir is a function of the instantaneous- 
field-of-view, 8 , of the scanner measured in milliradi- 
ans (mrad) and the altitude of the scanner above 
ground level, H, where D = B x H. As the scanner’s 
instantaneous-field-of-view moves away from nadir on 
either side, the circle becomes an ellipsoid because the 
distance from the aircraft to the resolution cell is in- 
creasing, as shown in Figure 7-2. In fact, the distance 
from the aircraft to the resolution cell on the ground, 
Hy, is a function of the scan angle off-nadir, , at the 
time of data collection and the true altitude of the air- 
craft, H (Lillesand et al., 2008): 


Hy, = Hx seco. (7.5) 


Vertical Aerial Photography Perspective Geometry 


PP 
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Line of aoe 
flight 
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Across-track Scanner Geometry with 
One-Dimensional Relief Displacement 
and Tangential Scale Distortion 


Scan direction 


ries 


Sigmoid distortion 
of linear feature 


Significant 
So geometric 
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<— scan line 
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FIGURE 7-3 a) Hypothetical perspective geometry of a vertical aerial photograph obtained over level terrain. Four 50-ft-tall 
tanks are distributed throughout the landscape and experience varying degrees of radial relief displacement the farther they 
are from the principal point (PP). b) Across-track scanning system introduces one-dimensional relief displacement perpendic- 
ular to the line of flight and tangential scale distortion and compression the farther the object is from nadir. Linear features 

trending across the terrain are often recorded with s-shaped or sigmoid curvature characteristics due to tangential scale dis- 


tortion and image compression. 


Thus, the size of the ground-resolution cell increases as 
the angle increases away from nadir. The nominal (av- 
erage) diameter of the elliptical resolution cell, Dy, at 
this angular location from nadir has the dimension: 


Dy = (Hx seco) xB (7.6) 
in the direction of the line of flight, and 
= (Hx seco) x B (7.7) 


in the orthogonal (perpendicular) scanning direction. 


Scientists using across-track scanner data usually con- 
cern themselves only with the spatial ground resolution 
of the cell at nadir, D. If it is necessary to perform pre- 
cise quantitative work on pixels some angle $ off-na- 
dir, then it may be important to remember that the 
radiant flux recorded is an integration of the radiant 
flux from all the surface materials in a ground-resolu- 
tion cell with a constantly changing diameter. Using 
only the central 70% of the swath width reduces the im- 
pact of the larger pixels found at the extreme edges of 
the swath. 


Scanning System One-Dimensional Relief 
Displacement 


Truly vertical aerial photographs have a single princi- 
pal point directly beneath the aircraft at nadir at the in- 
stant of exposure. This perspective geometry causes all 
objects that rise above the local terrain elevation to be 


displaced from their proper planimetric position radi- 
ally outward from the principal point. For example, 
the four hypothetical tanks in Figure 7-3a are each 50 
ft. high. The greater the distance from the principal 
point, the greater the radial relief displacement of the 
top of the tank away from its base. 


Images acquired using an across-track scanning system 
also contain relief displacement. However, instead of 
being radial from a single principal point, the displace- 
ment takes place in a direction that is perpendicular to 
the line of flight for each and every scan line, as shown 
in Figure 7-3b. In effect, the ground-resolution element 
at nadir functions like a principal point for each scan 
line. At nadir, the scanning system looks directly down 
on the tank, and it appears as a perfect circle in Figure 
7-3b. The greater the height of the object above the lo- 
cal terrain and the greater the distance of the top of the 
object from nadir (i.e., the line of flight), the greater the 
amount of one-dimensional relief displacement present. 
One-dimensional relief displacement is introduced in 
both directions away from nadir for each sweep of the 
across-track mirror. 


Although some aspects of one-dimensional relief dis- 
placement may be of utility for visual image interpreta- 
tion, it seriously displaces the tops of objects projecting 
above the local terrain from their true planimetric posi- 
tion. Maps produced from such imagery contain seri- 
ous planimetric errors. 


240 INTRODUCTORY DIGITAL IMAGE PROCESSING 


Scanning System Tangential Scale Distortion 
The mirror on an across-track scanning system rotates 
at a constant speed and typically views from 70° to 
120° of terrain during a complete line scan. Of course, 
the amount depends on the specific sensor system. 
From Figure 7-2 it is clear that the terrain directly be- 
neath the aircraft (at nadir) is closer to the aircraft 
than the terrain at the edges during a single sweep of 
the mirror. Therefore, because the mirror rotates at a 
constant rate, the sensor scans a shorter geographic 
distance at nadir than it does at the edge of the image. 
This relationship tends to compress features along an 
axis that is perpendicular to the line of flight. The 
greater the distance of the ground-resolution cell from 
nadir, the greater the image scale compression. This is 
called tangential scale distortion. Objects near nadir ex- 
hibit their proper shape. Objects near the edge of the 
flightline become compressed and their shape distort- 
ed. For example, consider the tangential geometric dis- 
tortion and compression of the circular swimming 
pools and one hectare of land the farther they are from 
nadir in the hypothetical diagram (Figure 7-3b). 


This tangential scale distortion and compression in the 
far range also causes linear features such as roads, rail- 
roads, utility rights-of-way etc., to have an s-shape or 
sigmoid distortion when recorded on scanner imagery 
(Figure 7-3b). Interestingly, if the linear feature is par- 
allel with or perpendicular to the line of flight, it does 
not experience sigmoid distortion. 


Even single flightlines of aircraft MSS data are difficult 
to rectify to a standard map projection because of air- 
craft roll, pitch, and yaw during data collection (van 
der Meer et al., 2009). Notches in the edge of a flight- 
line of data are indicative of aircraft roll. Such data re- 
quire significant human and machine resources to 
make the data planimetrically accurate. Most commer- 
cial data providers now place GPS on the aircraft to 
obtain precise flightline coordinates, which are useful 
when rectifying the aircraft MSS data. 


External Geometric Errors 


External geometric errors are usually introduced by 
phenomena that vary in nature through space and 
time. The most important external variables that can 
cause geometric error in remote sensor data are ran- 
dom movements by the aircraft (or spacecraft) at the 
exact time of data collection, which usually involve: 


* altitude changes, and/or 
* attitude changes (roll, pitch, and yaw). 


Altitude Changes 


A remote sensing system is ideally flown at a constant 
altitude above ground level (AGL) resulting in imagery 


with a uniform scale all along the flightline. For exam- 
ple, a frame camera with a 12-in. focal length lens 
flown at 20,000 ft. AGL will yield 1:20,000-scale imag- 
ery. If the aircraft or spacecraft gradually changes its 
altitude along a flightline, then the scale of the imagery 
will change (Figure 7-4a). Increasing the altitude will 
result in smaller-scale imagery (e.g., 1:25,000-scale). 
Decreasing the altitude of the sensor system will result 
in larger-scale imagery (e.g., 1:15,000). The same rela- 
tionship holds true for digital remote sensing systems 
that collect imagery on a pixel by pixel basis. The di- 
ameter of the spot size on the ground (D; the nominal 
spatial resolution) is a function of the instantaneous- 
field-of-view (f) and the altitude above ground level 
(H) of the sensor system, ie., D = Bx H. 


It is important to remember, however, that scale chang- 
es can be introduced into the imagery even when the 
remote sensing system is flown at a constant elevation 
above ground level. This occurs when the terrain grad- 
ually increases or decreases in elevation (i.e., it moves 
closer to or farther away from the sensor system). For 
example, if the terrain surface 1s at 1,000 ft. AGL at the 
start of a flightline and 2,000 ft. AGL at the end of the 
flightline, then the scale of the imagery will become 
larger as the flightline progresses. Remote sensing plat- 
forms do not generally attempt to adjust for such grad- 
ual changes in elevation. Rather, it is acknowledged 
that scale changes will exist in the imagery and that the 
use of geometric rectification algorithms will normally 
be used to minimize the effects. The methods for ad- 
justing for changes in scale using ground control points 
and geometric rectification coefficients will be dis- 
cussed shortly. 


Attitude Changes 

Satellite platforms are usually stable because they are 
not buffeted by atmospheric turbulence or wind. Con- 
versely, aircraft flying at suborbital altitudes must con- 
tinuously contend with atmospheric downdrafts, 
updrafts, head-winds, tail-winds, and cross-winds when 
collecting remote sensor data. Even when the remote 
sensing platform maintains a constant altitude AGL, it 
may rotate randomly about three separate axes that are 
commonly referred to as roll, pitch, and yaw (Figure 7- 
4b). For example, sometimes the fuselage remains hori- 
zontal, but the aircraft rolls from side to side about the 
x-axis (direction of flight) some w° , introducing com- 
pression and/or expansion of the imagery in the near- 
and far-ranges perpendicular to the line of flight (Wolf 
et al., 2013). Similarly, the aircraft may be flying in the 
intended direction but the nose pitches up or down a 
certain 6° about the y-axis. If the nose pitches down, 
the imagery will be compressed in the fore-direction 
(toward the nose of the aircraft) and expanded in the 
aft-direction (toward the tail). If the nose pitches up, 
the imagery will be compressed in the aft-direction and 
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Geometric Modification of Remotely Sensed Data 
Caused by Changes in Platform Altitude and Attitude 


a. Change in Altitude Decrease in altitude 
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FIGURE 7-4 a) Geometric modification in imagery may be introduced by changes in the aircraft or satellite platform altitude 
above ground level (AGL) at the time of data collection. Increasing altitude results in smaller-scale imagery while decreasing 
altitude results in larger-scale imagery. b) Geometric modification may also be introduced by aircraft or spacecraft changes in 
attitude, including roll, pitch, and yaw. An aircraft flies in the x-direction. Roll occurs when the aircraft or spacecraft fuselage 
maintains directional stability but the wings move up or down, i.e. they rotate about the x-axis angle (omega:  ). Aircraft 

pitch occurs when the wings are stable but the fuselage nose or tail moves up or down, i.e., they rotate about the y-axis an- 
gle (chi: o ). Yaw occurs when the wings remain parallel but the fuselage is forced by wind to be oriented some angle to the 
left or right of the intended line of flight, i.e., it rotates about the z-axis angle (kappa: « ). Thus, the plane flies straight but all 
remote sensor data are displaced by k° . Remote sensing data often are distorted due to a combination of changes in alti- 


tude and attitude (roll, pitch, and yaw). 
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expanded in the fore-direction. Occasionally a remote 
sensing platform experiences significant headwinds (or 
tailwinds) that must be compensated for in order to fly 
in a straight direction. When this occurs, it is possible 
for the pilot to “crab” (i.e., angle) the aircraft fuselage 
into the wind «° about the z-axis. The result is an ac- 
curate flightline but imagery that is oriented some «° 
from the intended flightline (Figure 7-4b). 


High-quality satellite and aircraft remote sensing sys- 
tems often have gyro-stabilization equipment that, in 
effect, isolates the sensor system from the roll and pitch 
movements of the aircraft. Remote sensing systems 
without stabilization equipment introduce some geo- 
metric error into the remote sensing dataset through 
variations in roll, pitch, and yaw that can only be cor- 
rected using ground control points. 


Ground Control Points 

Geometric distortions introduced by sensor system at- 
titude (roll, pitch, and yaw) and/or altitude changes 
can be corrected using ground control points and ap- 
propriate mathematical models (e.g., Im et al., 2009). A 
ground control point (GCP) is a location on the surface 
of the Earth (e.g., a road intersection) that can be iden- 
tified on the imagery and located accurately on a map. 
The image analyst must be able to obtain two distinct 
sets of coordinates associated with each GCP: 


* image coordinates specified in 7 rows and j columns, 
and 


* map coordinates (eg., x, y measured in degrees of 
latitude and longitude, feet in a state plane coordi- 
nate system, or meters in a Universal Transverse 
Mercator projection). 


The paired coordinates (i, j and x, y) from many GCPs 
(e.g., 20) can be modeled to derive geometric transfor- 
mation coefficients (Wolf et al., 2013). These coeffi- 
cients may then be used to geometrically rectify the 
remotely sensed data to a standard datum and map 
projection. 


Types of Geometric 
Correction 


Most commercially available remote sensor data (e.g., 
from SPOT Image Inc., DigitalGlobe Inc., GeoEye, 
Inc.) already have much of the systematic geometric er- 
ror removed. Unless otherwise processed, however, the 
unsystematic random error remains in the image, mak- 
ing it nonplanimetric (i.e., the pixels are not in their 
correct x, y planimetric map position). This section fo- 
cuses on two common geometric correction procedures 


often used by scientists to make the digital remote sen- 
sor data of value: 


* image-to-map rectification, and 
* image-to-image registration. 


The general rule of thumb is to rectify remotely sensed 
data to a standard map projection whereby it may be 
used in conjunction with other spatial information in a 
GIS to solve problems (Merchant and Narumalani, 
2009; Purkis and Klemas, 2011). Therefore, most of the 
discussion will focus on image-to-map rectification. 


Image-to-Map Rectification 


Image-to-map rectification is the process by which the 
geometry of an image is made planimetric. Whenever 
accurate area, direction, and distance measurements 
are required, image-to-map geometric rectification 
should be performed. It may not, however, remove all 
the distortion caused by topographic relief displace- 
ment in images. The image-to-map rectification pro- 
cess normally involves selecting GCP image pixel 
coordinates (row and column) with their map coordi- 
nate counterparts (e.g., meters northing and easting in 
a Universal Transverse Mercator map projection). For 
example, Figure 7-5 displays three GCPs (13, 14, and 
16) easily identifiable by an image analyst in both the 
U.S. Geological Survey 1:24,000-scale 7.5-minute 
quadrangle and an unrectified Landsat TM band 4 im- 
age of Charleston, SC. It will be demonstrated how the 
mathematical relationship between the image coordi- 
nates and map coordinates of the selected GCPs is 
computed and how the image is made to fit the geome- 
try of the map. 


In the U.S., there are several alternatives to obtaining 
accurate GCP map coordinate information for image- 
to-map rectification, including: 


¢ hard-copy planimetric maps (e.g., U.S. Geological 
Survey 7.5-minute 1:24,000-scale topographic maps) 
where GCP coordinates are extracted using simple 
ruler measurements or a coordinate digitizer; 


¢ digital planimetric maps (e.g., the U.S. Geological 
Survey’s digital 7.5-minute topographic map series) 
where GCP coordinates are extracted directly from 
the digital map on the screen; 


¢ digital orthophotoquads that are already geometri- 
cally rectified (e.g., U.S. Geological Survey digital 
orthophoto quarter quadrangles —DOQQ); and/or 
¢ global positioning system (GPS) instruments that 
may be taken into the field to obtain the coordinates 
of objects to within +20 cm if the GPS data are dif- 
ferentially corrected (e.g., Jensen and Jensen, 2013). 
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Selecting Ground Control Points for Image-to-Map Rectification 


yi 


x 


a. U. S. Geological Survey 7.5-minute 1:24,000-scale 
topographic map of Charleston, SC, with three 
ground control points identified. 


column (x) 


b. Unrectified Landsat Thematic Mapper band 4 
image obtained on November 9, 1982. 


FIGURE 7-5 Example of image-to-map rectification. a) U.S. Geological Survey 7.5-minute 1:24,000-scale topographic map of 
Charleston, SC, with three ground control points identified (13, 14, and 16). The GCP map coordinates are measured in me- 
ters easting (x) and northing (y) in a Universal Transverse Mercator projection. b) Unrectified 11/09/82 Landsat TM band 4 im- 
age with the three ground control points identified. The image GCP coordinates are measured in rows and columns. Image 


courtesy of NASA. 


GPS collection of map coordinate information to be 
used for image rectification is especially effective in 
poorly mapped regions of the world or where rapid 
change has made existing maps obsolete (Jensen et al., 
2002). 


Image-to-Ilmage Registration 


Image-to-image registration is the translation and rota- 
tion alignment process by which two images of like ge- 
ometry and of the same geographic area are positioned 
coincident with respect to one another so that corre- 
sponding elements of the same ground area appear in 
the same place on the registered images. This type of 
geometric correction is used when it is not necessary to 
have each pixel assigned a unique x, y coordinate in a 
map projection. For example, we might want to make 
a cursory examination of two images obtained on dif- 
ferent dates to see if any change has taken place. While 
it is possible to rectify both of the images to a standard 
map projection and then evaluate them (image-to-map 
rectification), this may not be necessary to simply 
identify the change that has taken place between the 
two images. 


Hybrid Approach to Image 
Rectification/Registration 


It is interesting that the same general image processing 
principles are used in both image rectification and im- 
age registration. The difference is that in image-to-map 
rectification the reference is a map in a standard map 
projection, while in image-to-image registration the 
reference is another image. It should be obvious that if 
a rectified image is used as the reference base (rather 
than a traditional map) any image registered to it will 
inherit the geometric errors existing in the reference 
image. Because of this characteristic, most serious 
Earth science remote sensing research is based on anal- 
ysis of data that have been rectified to a map base. 
However, when conducting rigorous change detection 
between two or more dates of remotely sensed data, it 
may be useful to select a Aybrid approach involving 
both image-to-map rectification and image-to-image 
registration (Jensen et al., 1993). 


An example of the hybrid approach is demonstrated in 
Figure 7-6 where an October 14, 1987, Landsat TM 
image is being registered to a rectified November 9, 
1982, Landsat TM scene. In this case, the 1982 base 
year image was previously rectified to a Universal 
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Transverse Mercator map projection with 30 X 30 m 
pixels. Ground control points are being selected to reg- 
ister the 1987 image to the rectified 1982 base year im- 
age. It is often very difficult to locate good ground 
control points in remotely sensed data, especially in ru- 
ral areas (e.g., forests, wetland, and water bodies). The 
use of the rectified base year image as the map allows 
many more common GCPs to be located in the unrec- 
tified 1987 imagery. For example, edges of water bodies 
and fields or the intersection of small stream segments 
are not usually found on a map but may be easy to 
identify in the rectified and unrectified imagery. 


The optimum method of selecting the ground control 
points is to have both the rectified base year image (or 
reference map) and the image to be rectified on the 
screen at the same time (Figure 7-6). This dual display 
greatly simplifies GCP selection. Some image process- 
ing systems even allow the GCP selected to be repro- 
jected onto the image to be corrected (with the 
appropriate transformation coefficients, to be dis- 
cussed shortly) to determine the quality of the GCP 
point. Also, some systems allow the analyst to extract 
floating point row and column coordinates of GCPs 
(instead of just integer values) through the use of a 
chip extraction algorithm that zooms in and does sub- 
pixel sampling, as demonstrated in Figure 7-6. GCP 
subpixel row and column coordinates often improve 
the precision of the image-to-map rectification or im- 
age-to-image registration. Some scientists have devel- 
oped analytical methods of automatically extracting 
GCPs common to two images that can be used during 
image-to-image registration. However, most image-to- 
map rectification still relies heavily on human selection 
of GCPs. 


The following example focuses on image-to-map geo- 
metric rectification because it is the most frequently 
used method of removing geometric distortion from re- 
motely sensed data. 


Image-to-Map Geometric 
Rectification Logic 


Two basic operations must be performed to geometri- 
cally rectify a remotely sensed image to a map coordi- 
nate system: 


1. The geometric relationship between the input pixel 
coordinates (column and row; referred to as x’, y’) 
and the associated map coordinates of this same point 
(x, y) must be identified (Figures 7-5 and 7-6). A 
number of GCP pairs are used to establish the nature 
of the geometric coordinate transformation that must 
be applied to rectify or fill every pixel in the output 
image (x,y) with a value from a pixel in the 


unrectified input image (x’, y’ ). This process is called 
spatial interpolation. 


2. Pixel brightness values must be determined. 
Unfortunately, there is no direct one-to-one 
relationship between the movement of input pixel 
values to output pixel locations. It will be shown 
that a pixel in the rectified output image often 
requires a value from the input pixel grid that does 
not fall neatly on a row-and-column coordinate. 
When this occurs, there must be some mechanism 
for determining the brightness value (BV ) to be 
assigned to the output rectified pixel. This process 
is called intensity interpolation. 


Spatial Interpolation Using Coordinate 
Transformations 


As discussed earlier, some distortions in remotely 
sensed data may be removed or mitigated using tech- 
niques that model systematic orbital and sensor char- 
acteristics. Unfortunately, this does not remove error 
produced by changes in attitude (roll, pitch, and yaw) 
or altitude. Such errors are generally unsystematic and 
are often removed by identifying GCPs in the original 
imagery and on the reference map and then mathemat- 
ically modeling the geometric distortion. Image-to- 
map rectification requires that polynomial equations 
be fit to the GCP data using least-squares criteria to 
model the corrections directly in the image domain 
without explicitly identifying the source of the distor- 
tion (Novak, 1992; Bossler, 2010). Depending on the 
distortion in the imagery, the number of GCPs used, 
and the degree of topographic relief displacement in 
the area, higher-order polynomial equations may be re- 
quired to geometrically correct the data. The order of 
the rectification is simply the highest exponent used in 
the polynomial. For example, Figure 7-7 demonstrates 
how different-order transformations fit a hypothetical 
surface. Generally, for moderate distortions in a rela- 
tively small area of an image (e.g., a quarter of a Land- 
sat TM scene), a first-order, six-parameter, affine 
(linear) transformation is sufficient to rectify the imag- 
ery to a geographic frame of reference. 


This type of transformation can model six kinds of dis- 
tortion in the remote sensor data, including (Novak, 
1992; Buiten and Van Putten, 1997): 


* translation in x and y, 

¢ scale changes in x and y, 
* skew, and 

* rotation. 


Input-to-Output (Forward) Mapping: When all six 
of the operations are combined into a single expression 
it becomes: 
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Image-to-Image Hybrid Registration 


a. Rectified Landsat TM band 4 image of 
Charleston, SC, obtained on November 9, 1982. 


b. Unrectified Landsat TM band 4 image 
obtained on October 14, 1987. 


FIGURE 7-6 Example of image-to-image hybrid registration. a) Previously rectified Landsat TM band 4 data obtained on 
November 9, 1982, resampled to 30 X 30 m pixels using nearest-neighbor resampling logic and a UTM map projection. 
b) Unrectified October 14, 1987, Landsat TM band 4 data to be registered to the rectified 1982 Landsat scene. Original 


imagery courtesy of NASA. 
(7.8) 


xX = dyta,x'+ayy’ 


y = bot Bx' + by’, 


where x and y are positions in the output-rectified im- 
age or map, and x’and y’ represent corresponding po- 
sitions in the original input image or map (Figure 7- 
8a). These two equations can be used to perform what 
is commonly referred to as input-to-output or forward- 
mapping. The equations function according to the logic 
shown in Figure 7-8a. In this example, each pixel in the 
input grid (e.g., value 15 at x’, y’ = 2, 3) is sent to an 
x,y location in the output image according to the six 
coefficients shown in Figure 7-8a. 


This forward mapping logic works well 1f we are simply 
rectifying the location of discrete coordinates found 
along a linear feature such as a road in a vector map. 
In fact, cartographic mapping and geographic infor- 
mation systems typically rectify vector data using for- 
ward mapping logic. However, when we are trying to 
fill a rectified output grid (matrix) with values from an 
unrectified input image, forward mapping logic does 
not work well. The basic problem is that the six coeffi- 
cients may require that value 15 from the x’, y’ loca- 
tion 2,3 in the input image be located at a floating 
point location in the output image at x, y = 5, 3.5, as 


shown in Figure 7-8a. The output x, y location does 
not fall exactly on an integer x and y output map coor- 
dinate. In fact, using forward mapping logic can result in 
output matrix pixels with no output value (Wolberg, 
1990). This is a serious condition and one that reduces 
the utility of the remote sensor data for useful applica- 
tions. For this reason, most remotely sensed data are 
geometrically rectified using outputto-inputor inverse 
mapping logic (Niblack, 1986; Richards, 2013). 


Output-to-Input (Inverse) Mapping: The use of 
output-to-input, or inverse mapping logic, 1s based on 
the following two equations: 


x' = ajtayxtany 


(7.9) 


y' = byt bx + by 


where x and y are positions in the output-rectified im- 
age or map, and x’ and y’ represent corresponding 
positions in the original input image. If desired, this re- 
lationship can also be written in matrix notation: 


| - ealldG} 


(7.10) 
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Observations 


Linear: y = b + ax 


Cubic: y = d + cx + bx? + ax3 


FIGURE 7-7 Concept of how different-order transformations fit a hypothetical surface illustrated in cross-section. a) The orig- 
inal observations. b) A first-order linear transformation fits a plane to the data. c) A second-order quadratic fit. d) A third-order 


cubic fit. 


The rectified output matrix consisting of x (column) 
and y (row) coordinates is filled in the following sys- 
tematic manner. Each output pixel location (e.g., x, y = 
5, 4 in Figure 7-8b) is entered into Equation 7.9. The 
equation uses the six coefficients to determine where to 
go into the original input image to get a value (dashed 
line in Figure 7-8b). In this example, it requests a value 
from the floating point location x’, y' = 2.4, 2.7. Inter- 
estingly, there is no value at this location. Nevertheless, 
it is possible to get a value (e.g., 15) if nearest-neighbor 
resampling logic is used (to be discussed shortly). The 
inverse mapping logic guarantees that there will be a 
value at every x, y coordinate (column and row) in the 
output image matrix. There will be no missing values. 
This procedure may seem backwards at first, but it is 
the only mapping function that avoids overlapping pix- 
els and holes in the output image matrix (Schoweng- 
erdt, 2007; Konecny, 2014). 


Sometimes it is difficult to gain an appreciation of ex- 
actly what impact the six coefficients in an affine (lin- 
ear) transformation have on geographic coordinates. 
Therefore, consider Table 7-1, which demonstrates 
how the six coefficients (in parentheses) in a forward 


affine transformation influence the x, y coordinates of 
16 pixels in a hypothetical 4 X 4 matrix. Note how 
translation (shifting) in x and y is controlled by the ag 
and bp coefficients, respectively. Scale changes in x and 
y are controlled by the a, and b> coefficients, respec- 
tively. Shear/rotation in x and y are controlled by coef- 
ficients a, and by, respectively. The three examples 
demonstrate how modifying the coefficients causes the 
pixel coordinates to be translated (shifted one unit in x 
and y), scaled (expanded by a factor of 2 in x and y), 
and sheared/rotated (Brown, 1992). Please remember 
that this particular example is based on forward map- 
ping logic. 


It is possible to use higher-order polynomial transfor- 
mations to rectify remotely sensed data. For example, 
instead of using the six-parameter affine transforma- 
tion previously discussed, we could use a second-order 
(quadratic) polynomial: 


2 2 
x’ = eg text ecgytesxy +eyx + csy 


(7.11) 


yl =dyotdxtdyy+dsxyt dy td’. (7.12) 
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fa 2 Input-to-Output (Forward) Mapping Logic 
"ete { 3 i 2 3g 4 § 6 
/ nae 
-~ 
: 2 
, 3 ~l4 ~— / 4 Take the value 15 at input location 2, 3 and 
J / (715) = place it in the output image at 5, 3.5 
/ 8 RE 
4 ~14 17 3 5 
[~~ 
S~15 i 29 4 
gp / 
/ 20 / 
6-16 i / gy 5 
t ‘ — ™> 2 6 a. 
ae 
Original input image (x', y') Rectified output image (x, y) 
X=ayta,x'tay' 
y=byt+b,x'+b,y' where 
x' and y' are locations in the original input image, and 
x and y are locations in the rectified output image. 
id 5 24 Output-to-Input (Inverse) Mapping Logic 
I - ul / 
fies |S , 1 2 3 4 ~ 5 6 


Go into the input image at 2.4, 2.7 and 
get a value for output location 5, 4. 
a 


Original input image (x', y') Rectified output image (x, y) 


x'=ayta,;x tay 


FIGURE 7-8 a) The logic of filling a rectified out put matrix with values from an unrectified input image matrix using input-to- 
output (forward) mapping logic. b) The logic of filling a rectified output matrix w th values from an unrectified input image 
matrix using output-to-input (inverse) mapping logic and nearest-neighbor resampling. Output-to-input inverse mapping 
logic is the preferred methodology because it results in a rectified output matrix with values at every pixel location. 
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TABLE 7-1 Forward mapping example of how varying the six coefficients (in parentheses) in a linear (affine) transformation 


can impact the coordinates of 16 pixels in a simple 44 4 matrix. The equations are of the form x 


predict = 49 + 44x + any and 


Voda = by + b,x + boy. Translation (shfting) in x and y is controlled by the ag and bg coefficients, respectively. Scale 
changes in x and y are controlled by the a, and by, coefficients, respectively. Shear/rotation in x and y are controlled by coeffi- 


cients az and by, respectively. 


Original Matrix Translation Scale Shear/Rotation 
Xpredict = (0) + (1)x + (O)y Xpredict = (1) + (1)x + (O)y Xpredict = (0) + (2)x + (O)y Xpredict = (0) + (1)x + (2)y 
Ypredict = (0) + (O)x + (Ty | Ypredict = (1) + (O)x + (1)¥ | Ypredict = (0) + (0)x + (2)y Ypredict = (0) + (2)x + (1)y 
0123456789 1NL 012345678 912 ol12a45678 912 ol123245 678 912 

0 . 0 0 ps 0 . 
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5 5 0000 5 5 ® @ 
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In theory, the higher the order of the polynomial, the 
more closely the coefficients should model the geomet- 
ric error in the original (unrectified) input image (e.g., 
refer to Figure 7-7 to see how the various models fit 
one-dimensional data) and place the pixels in their cor- 
rect planimetric positions in the rectified output ma- 
trix. Higher-order polynomials often produce a more 
accurate fit for areas immediately surrounding ground 
control points. However, other geometric errors may be 
introduced at large distances from the GCPs (Gibson 
and Power, 2000). In addition, the digital image pro- 


cessing system time required to geometrically rectify 
the remote sensor data using higher-order polynomials 
increases because of the greater number of mathemati- 
cal operations that must be performed. 


A general rule of thumb is to use a first-order affine 
polynomial whenever possible. Select a higher-order 
polynomial (e.g., second or third order) only when 
there are serious geometric errors in the dataset. These 
types of error are often found in imagery obtained 
from suborbital aerial platforms where roll, pitch, and 
yaw by the aircraft introduce unsystematic, nonlinear 
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Geometric Correction of Airborne Imaging Spectrometer for Applications 
iat Imagery of Forest Experimental Plots near Aiken, SC 
0 ie: eh ; . 


ef a 


b. Hyperspectral data geometrically corrected using GCPs, a third-order 
polynomial and nearest-neighbor resampling. 


FIGURE 7-9 a) AISA 1 1 m hyperspectral data collected on September 15, 2006. Only three of the 63 channels are dis- 
played (RGB = bands 760.8, 664.4, and 572.5 nm). b) The 63-channel dataset was rectified using a second-order polynomial 
to adjust for the significant geometric distortion in the original dataset caused by the turbulence during data collection. The 
forest experimental plots are subjected to various nutrient (fertilizer) and irrigation treatments (based on Im, J., Jensen, J. R., 
Coleman, M. and E. Nelson, 2009, “Hyperspectral Remote Sensing Analysis of Short Rotation Woody Crops Grown with Con- 
trolled Nutrient and Irrigation Treatments," Geocarto International, 24(4):293-312). 
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distortions. These distortions can only be modeled us- 
ing a higher-order polynomial (e.g., Im et al., 2009). 
For example, the Airborne Imaging Spectrometer for 
Applications (AISA) hyperspectral imagery of forest 
experimental treatment plots shown in Figure 7-9 was 
geometrically rectified using a third-order polynomial. 
A first-order affine rectification was not sufficient to 
correct the significant warping in the imagery caused 
by the atmospheric turbulence during data collection. 
All pixels in this image are now in their appropriate 
geographic location (Im et al., 2009). Note that the 
roads in the rectified image are now straight. 


Compute the Root-Mean-Squared Error of the 
Inverse Mapping Function Using the six coordinate 
transform coefficients that model distortions in the 
original scene, it is possible to use the output-to-input 
(inverse) mapping logic to transfer (relocate) pixel val- 
ues from the original distorted image x’, y’ to the grid 
of the rectified output image, x, vy. However, before ap- 
plying the coefficients to create the rectified output im- 
age, it is important to determine how well the six 
coefficients derived from the least-squares regression of 
the initial GCPs account for the geometric distortion 
in the input image. The method used most often in- 
volves the computation of the root-mean-square error 
(RMSgrror) for each of the ground control points (Wolf 
et al., 2013). 


Let us consider for a moment the nature of the GCP 
data. We first identify a point in the image such as a 
road intersection. Its column and row coordinates in 
the original input image we will call xo,jg and Yorig. The 
x and y position of the same road intersection is then 
measured from the reference map in degrees, feet, or 
meters. These two sets of GCP coordinates and many 
others selected by the analyst are used to compute the 
six coefficients discussed in Equation 7.9. Now, if we 
were to put the map x and y values for the first GCP 
back into Equation 7.9, with all the coefficients in 
place, we would get computed x’ and y’ values that 
are supposed to be the location of this point in the in- 
put image space. Ideally, x’ would equal x,,j¢ and y’ 
would equal yo,js. Unfortunately, this is rarely the case. 
Any discrepancy between the values represents image 
geometric distortion not corrected by the six-coeffi- 
cient coordinate transformation. 


A simple way to measure such distortion is to compute 
the RMS,,;0, for each ground control point by using 
the equation: 


RMB woe = l(a" = nee + (y' Voss) ’ (7. 13) 


where Xorjg aNd Vorig are the original row and column 
coordinates of the GCP in the image and x’ and y’ are 
the computed or estimated coordinates in the original 


image. The square root of the squared deviations repre- 
sents a measure of the accuracy of this GCP in the im- 
age. By computing RMS,,+0; for all GCPs, it is possible 
to 1) see which GCPs exhibit the greatest error, and 2) 
sum all the RMS.,+0;- 


Normally, the user specifies a certain amount (a 
threshold) of acceptable total RMS.,;0; (e.g., 1 pixel). 
If an evaluation of the total RMSero, reveals that a 
given set of control points exceeds this threshold, it is 
common practice to 1) delete the GCP that has the 
greatest amount of individual error from the analysis, 
2) recompute the six coefficients, and 3) recompute the 
RMSorror for all points. This process continues until 
one of the following occurs: the total RMSg,ro; 1s less 
than the threshold specified or too few points remain 
to perform a least-squares regression to compute the 
coefficients. Once the acceptable RMSeryo, is reached, 
the analyst can proceed to the intensity interpolation 
phase of geometric rectification, which attempts to fill 
an output grid (x, y) with brightness values found in 
the original input grid (x’, y’ ). 


Intensity Interpolation 

The intensity interpolation process involves the extrac- 
tion of a brightness value from an x’, y’ location in the 
original (distorted) input image and its relocation to 
the appropriate x, y coordinate location in the rectified 
output image. This pixel-filling logic is used to produce 
the output image line by line, column by column. Most 
of the time the x’ and y’ coordinates to be sampled in 
the input image are floating point numbers (i.e., they 
are not integers). For example, in Figure 7-8b we see 
that pixel 5, 4 (x, y) in the output image is to be filled 
with the value from coordinates 2.4, 2.7 (x', y’) in the 
original input image. When this occurs, there are sever- 
al methods of brightness value (BV) interpolation that 
can be applied, including: 


* nearest neighbor, 
¢ bilinear interpolation, and 
* cubic convolution. 


The practice is commonly referred to as resampling. 


Nearest-neighbor Interpolation: When an image 
analyst uses nearest-neighbor interpolation, the bright- 
ness value closest to the specified x’, y’ coordinate is 
assigned to the output x, y coordinate. For example, in 
Figure 7-8b, the output pixel 5, 4 (x, y) requests the 
brightness value in the original input image at location 
2.4, 2.7 (x', v'). There is no value at this location. 
However, there are nearby values at the integer grid in- 
tersections. Distances from 2.4, 2.7 (x', vy’) to neigh- 
boring pixels are computed using the Pythagorean 


CHAPTER 7 GEOMETRIC CORRECTION 251 


TABLE 7-2 Bilinear interpolation of a weighted brightness value (BV,,,) at location x’, y’ based on the analysis of four sample 


points in Figure 7-8b. 


theorem. A nearest-neighbor rule would assign the 
output pixel (x, y) the value of 15, which is the value 
found at the nearest input pixel. 


This is a computationally efficient procedure. It is espe- 
cially liked by Earth scientists because it does not alter 
the pixel brightness values during resampling. It is of- 
ten the very subtle changes in brightness values that 
make all the difference when distinguishing one type of 
vegetation from another, an edge associated with a 
geologic lineament, or different levels of turbidity, 
chlorophyll, or temperature in a lake. Other interpola- 
tion techniques to be discussed use averages to com- 
pute the output intensity value, often removing 
valuable spectral information. Nearest-neighbor resa- 
mpling should be used whenever biophysical informa- 
tion is to be extracted from remote sensor data. 


Bilinear Interpolation: First-order or bilinear inter- 
polation assigns output pixel values by interpolating 
brightness values in two orthogonal directions in the 
input image. It basically fits a plane to the four pixel 
values nearest to the desired position (x’, y’) in the in- 
put image and then computes a new brightness value 
based on the weighted distances to these points. For 
example, the distances from the requested x’, y’ posi- 
tion at 2.4, 2.7 in the input image in Figure 7-8b to the 
closest four input pixel coordinates (2, 2; 3, 2; 2, 3; 3, 3) 
are computed in Table 7-2. The closer a pixel is to the 
desired x’, y’ location, the more weight it will have in 
the final computation of the average. The weighted av- 
erage of the new brightness value (BY,,,,) is computed 

using the equation: 7 Zz 

Di 

eqs k=1 

Bilinear gy = ; 


(7.14) 


Sample Point Value at Z 1 
Location Sample Distance from x’, y’ D? D? 
(column, row) Point, Z to the Sample Point, D D2 
2, 2 9) 18: 14559) 
p = \(24-2)%+(27-2)* = 0.806 oe? 7e : 
3,2 6 0.85 7.06 1.176 
p = \(24~3)%+(27-2)* -0.921 
2 1 we. : 4. 
ie ; D = .(2.4—2)* +(2.7-3)* = 0.500 ee ean ye 
3,3 18 0.45 40.00 2.222 
p = \(24-3)*+(27-3)* =0.670 
3 IZA0S| 8.937 
BV, = 120.91/8.937 
=13.53 


where Z, are the surrounding four data point values, 
and D? are the distances squared from the point in 
question (x', y’) to these data points. In our example, 
the weighted average of BV,,, is 13.53 (truncated to 
13), as shown in Table 7-2. The average without 
weighting is 12. In many respects this method acts as a 
spatial moving filter that subdues extremes in bright- 
ness value throughout the output image. 


Cubic Convolution: Resampling assigns values to 
output pixels in much the same manner as bilinear in- 
terpolation, except that the weighted values of sixteen 
input pixels surrounding the location of the desired 
x', yy’ pixel are used to determine the value of the out- 
put pixel. For example, the distances from the request- 
ed x',y’ position at 2.4,2.7 in the input image in 
Figure 7-8b to the closest 16 input pixel coordinates 
are computed in Table 7-3. The weighted average of the 
new brightness value (BV,,,) is computed using the 
equation: 


Cubic Convolution ;; = : : (7.15) 


where Z; are the surrounding 16 data point values, and 
D? are the distances squared from the point in ques- 
tion (x', y’) to these data points. In this example, the 
weighted average of BV,,, is 13.41 (truncated to 13), as 
shown in Table 7-3. The average without weighting af- 
ter truncation is 12. 
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TABLE 7-3 Cubic convolution interpolation of a weighted brightness value (BV,,,) at location x’, y’ based on the analysis of 
16 sample points in Figure 7-8b. 


Sample Point Value at Z 1 
Location Sample Distance from x’, y’ D? as a 
(column, row) Point, Z to the Sample Point, D Dz DR 
ly 1 7 2 5) 4.85 1.443 0.206 
De seamen rely. = 2202 
2,1 8 5) 5} 3.05 2.623 0.328 
D= dh2.a—2) +(2.7-1)* = 1.746 
Saul 6 2 5) 3.24 1.852 0.309 
De de2a—3) +(2.7-1)* = 1.80 
4,1 Yi 5} 3 5.45 1.284 0.183 
D = 4(2.4—4)2 + (27~-1)* = 2.335 
1, 2 8 2 2 2.45 3.265 0.408 
De 24-1) +(2.7-2)° = 1.565 
2,2 9 2 2 0.65 13.85 1.539 
D= dho.a—2) +(2.7-2)* =0.806 
3, 2 6 2 5} 0.85 7.06 1.176 
p = \(2.4-3)2+(27-2)* = 0.921 
4,2 14 2 3 3.05 4.59 0.328 
D = \(2.4—4)2 4 (2.7-2)% = 1.746 
1,3 14 2 2 2.05 6.829 0.488 
io) = d2a—1) +(2.7-3)° = 1.432 
2,3 15 2 3 0.25 60.00 4.000 
D= dhoa—2) +(2.7-3)° =0.500 
SHS 18 2 2 0.45 40.00 2.222 
De di24—3) +(2.7-3)° =0.670 
4,3 17 5) 5) 2.65 6.415 0.377 
D= daa) +(2.7-3)° = 1.63 
1,4 14 2 5) 3.65 3.836 0.274 
p= d2a—1) +(2.7-4)* =1.911 
2,4 16 2 3 1.85 8.649 0.541 
D = \(2.4-2)2 + (27-42 = 1.360 
3,4 19 2 2 2.05 9.268 0.488 
fo) = di24—3) +(2.7-4)° = 1.432 
4,4 20 2 2 4.25 4.706 0.235 
D= Ji2a—4) +(2.7-4)* = 2.062 
x 175.67 x 13.102 
BV wt = 175.67 / 13.102 
BV,,,= 13.41 
™ An Example of Image-to- dataset such as a Landsat Thematic Mapper image of 


Charleston, SC. The image-to-map rectification pro- 
cess generally involves: 


Ba Viap Rectification 


To appreciate digital image-to-map rectification, it is fe BEI ECS at aPPHoMale Dap PrOjechon, 


useful to demonstrate the logic by applying it toa real ° Collecting ground control points, 


¢ determining the optimum set of geometric rectifica- 
tion coefficients by iteratively computing the total 
GCP RMSerror, and 


¢ filling the output matrix using spatial and intensity 
interpolation resampling methods. 


Selecting an Appropriate Map 
Projection 


A map projection is a systematic transformation of the 
three-dimensional Earth into a two-dimensional flat 
map (Iliffe, 2008; Garnett, 2009). The characteristics of 
the map projection to which the remote sensor data 
will be rectified are a very important consideration 
(Merchant and Narumalani, 2009). Cartographers and 
mathematicians have devised a great variety of map 
projections to project (flatten) the three-dimensional 
characteristics of the globe onto a two-dimensional flat 
map (Robinson and Snyder, 1991; Maher, 2010). Many 
of the most useful map projections are summarized in 
John Snyder’s USGS Professional Paper Map Projec- 
tion: A Working Manual (1987) and in Flattening the 
Earth (1995). The following material identifies some of 
the key properties, characteristics, and preferred uses 
of several of the most important map projections fre- 
quently used by image analysts for image rectification. 


Developable Surfaces used to Create Map 
Projections 


Cartographers have developed methods to project the 
information found on the surface of a globe onto sim- 
ple geometric forms, called developable surfaces, in- 
cluding (Figure 7-10): 


¢ planes, 
¢ cylinders, and 
* cones. 


A developable surface is a simple geometric form capa- 
ble of being flattened without compressing or stretch- 
ing (Slocum et al., 2008). A plane is already flat, 
whereas a cylinder or cone may be cut and laid out flat, 
without stretching or compressing the content. Map 
projections are typically classified into three general 
families: cylindrical, conical, and azimuthal (planar). 


One of the most important characteristics of map pro- 
jections is where the developable surface just touches 
or intersects the sphere. For example, if the sphere 
(globe) just touches the developable surface at a single 
point, it is said to be tangent to the surface as shown in 
Figure 7-10a,c,e. If the sphere intersects the develop- 
able surface it is said to be secant to the surface as 
shown in Figure 7-10b,d,f). The place(s) where the 
sphere intersects the developable surface (i.¢., it is ei- 
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ther tangent or secant to the surface) is the most accu- 
rate area of a map projection. 


Map Projection Characteristics 

Unfortunately, the map projection process cannot be 
accomplished without some distortion, resulting in ev- 
ery map projection having unique advantages and dis- 
advantages (Grafarend and Krumm, 2006; Kanters, 
2007; Krygier, 2011) (Table 7-4). Consequently, there is 
no universal “best” map projection for each mapping 
project (USGS, 2011). Instead, the image analyst must 
select the map projection best suited to his or her 
needs, which minimizes the distortion of the most im- 
portant features to be portrayed on the map. 


Please remember that every two-dimensional flat map 
distorts the surface of the three-dimensional Earth in 
some way. A projected flat map or parts of a map can 
have some, but never all, of the following characteris- 
tics (Table 7-4): 


¢ true directions, 

* true distances (i.e., equidistant), 
* true areas, and/or 

¢ true shapes (1.e., conformality). 


A map projection is said to be conformal when at any 
point on the map the scale is the same in every direc- 
tion (USGS, 2011). Therefore, meridians and parallels 
in conformal map projections intersect at right angles 
and the shapes of very small areas and angles with very 
short sides are preserved. The size of most areas, how- 
ever, is distorted. Because angles at each point are cor- 
rect on conformal maps, the scale in every direction 
around any point is constant. This allows the analyst 
to measure distance and direction between relatively 
near points with good precision. For our purposes, this 
means that for image areas covering a few contiguous 
1:24,000-scale 7.5-minute quadrangle sheets, accurate 
spatial measurement is possible if the data are rectified 
to a conformal map projection. 


A map projection is equal-area (also referred to as 
equivalent) if every part on the map, as well as the 
whole, has the same area as the corresponding part on 
the Earth, at the same reduced scale. No flat map can 
be both equal-area and conformal at the same time 
(USGS, 2011). 


Equidistant maps show true distances only from the 
center of the projection or along a special set of lines 
(USGS, 2011). They correctly represent azimuths 
about the point of tangency (i.e., where a map touches 
the globe) but do not correctly represent areas. For ex- 
ample, an Azimuthal Equidistant map projection cen- 
tered on Washington, DC, shows the correct distance 
between Washington, DC, and any other point in the 
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FIGURE 7-10 Three types of developable surfaces are often used to create map projections: plane, cylinder, and cone. ab) A 
planar developable surface may just touch (i.e., be tangent with) or intersect (i.e., be secant with it) a sphere. cd) A cylindrical 
developable surface may be tangent or secant to a sphere. ef) A cone developable surface may be tangent or secant to a 


sphere (Jensen and Jensen, 2013). 


projection, i.e., it shows the correct distance between 
Washington, DC, and Philadelphia, PA, and between 
Washington, DC, and Richmond, VA. But it does not 
show the correct distance between Philadelphia, PA, 
and Richmond, VA, because the projection is not cen- 
tered on Philadelphia, PA. No flat map can be both 
equidistant and equal-area at the same time. A circle of 
diameter 1 drawn at any location on the map will en- 
compass exactly the same geographic area. This char- 
acteristic is useful if a scientist is interested in 
comparing land-use area, density, and so on. Unfortu- 
nately, to maintain the equal-area attribute, the shapes, 
angles, and scale in parts of the map may be distorted. 


Azimuthal map projections correctly represent selected 
angular relationships. Azimuthal projections are pro- 
jections to a plane placed tangent to (just touching) the 
globe at a point. As with distances, not all angular rela- 
tionships can be represented correctly on a single map, 
but it is possible to correctly represent all angular rela- 
tionships about a single point. A basic knowledge of 


the properties of commonly used map projections 
helps when selecting a map projection that comes clos- 
est to fulfilling a specific need. 


It is not possible to describe or show examples of all 
the map projections available (Esri, 2004; Intergraph, 
2013; Furuti, 2011). Rather, the following discussion 
focuses on just a few map projections that are especial- 
ly useful when rectifying remote sensor data. 


Cylindrical Map Projections 

Standard cylindrical map projections are mathemati- 
cally projected onto a cylinder that a) just touches the 
sphere (1.e., it is tangent to the sphere), or b) is made to 
intersect the sphere (i.e., it is secant to the sphere) (Fig- 
ure 7-1lab). 


Mercator Map Projection: The Mercator map pro- 
jection was developed by Flemish cartographer Gerar- 
dus Mercator (1512-1594) in 1569 for navigation 
purposes (Figure 7-12ab). Meridians and parallels are 
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TABLE 7-4 Summary of selected map projection proper ties associated w th cylindrical, azimuthal, and conic develop able sur- 


faces. P = partly (based on USGS, 2011). 


Developable Equal Equi- True Straight 
Surface Type Projection Conformal area distant Direction | Perspective | Rhumb lines 
Sphere Globe 
Cylindrical Mercator P 
Transverse Mercator 
Space Oblique 
Mercator 
Azimuthal Gnomonic P 
Stereographic P 
Orthographic P 
Azimuthal P P 
Equidistant 
Lambert Azimuthal P 
Equal-area 
Conic Albers Equal-area 
Lambert P 
Conformal 
Equidistant P 
Polyconic P 


straight lines and cross one another at 90M Angular re- 
lationships are maintained in this map projection. 
However, to preserve conformality, parallels of latitude 
are placed increasingly farther apart with increasing 
distance from the Equator (Figure 7-12ab). Any 
straight line on a Mercator map projection is a rhumb 
line (a line of constant angular direction), which is not 
necessarily the shortest distance between points. Dis- 
tances are true only along the Equator (i.e., the stan- 
dard parallel), but are reasonably correct within 12X to 
15X north or south of the Equator (Figure 7-1la). If 
the developable surface is secant to the sphere, two 
standard parallels will have the correct scale instead of 
the Equator (Figure 7-11b) (Slocum et al., 2008; Krygi- 
er, 2011). 


Areas and shapes of large areas like continents are usu- 
ally distorted in Mercator map projections. The distor- 
tion increases away from the Equator and there is 


extreme distortion in the polar regions. The poles are 
typically not shown in a Mercator map projection be- 
cause of the extreme distortion in these areas. A Mer- 
cator map projection of MODIS satellite image data 
and global elevation data are shown in Figure 7-12ab. 


The Mercator map projection is excellent for mapping 
equatorial regions. Otherwise, the Mercator projection 
should be considered a special-purpose map projection 
best suited for navigation. Secant Mercator map pro- 
jections are used for large-scale coastal charts. In fact, 
the use of the Mercator map projection for nautical 
charting is universal. Examples are the charts pub- 
lished by the National Ocean Survey, U.S. Dept. of 
Commerce (Intergraph, 2013). 


Mercator map projections are conformal in that angles 
and shapes within any small geographic area (such as 
that found in a typical USGS 7.5-minute topographic 
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FIGURE 7-11 Cylindrical map projection pattern of distortion where a) the developable surface just touches the sphere, i-e., 
it is tangent, and b) when it intersects the sphere, i.e. it is secant. cd) Tangent and secant azimuthal map projection patterns 
of distortion. ef) Tangent and secant conical map projection patterns of distortion (adapted from Jensen and Jensen, 2013). 
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Cylindrical Map Projections 
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FIGURE 7-12 a) A Mercator map projection of NASA Moderate Resolution Imaging Spectrometer (MODIS) satellite data. 

The Mercator conformal map projection is based on a cylindrical developable surface (refer to Figure 7-11ab). The Equator is 
often the standa rd parallel. Note the extreme distortion of land masses at higher latitudes. b) A Mercator conformal map pro- 
jection of global elevation data. The MODIS imagery and elevation data are courtesy of NASA. Both projections were created 


using NASA's G.Projector software. 


map) are essentially true (USGS, 2011). The patterns 
of distortion associated with cylindrical map projec- 
tions that just touch (tangent) or intersect (secant) the 
globe are shown in Figure 7-1lab (Slocum et al., 2008; 
Krygier, 2011). 


Universal Transverse Mercator (UTM) Projection: 
One of the most widely used map projections for recti- 
fying remotely sensed data and for large-scale topo- 


graphic mapping applications is the Transverse 
Mercator projection. It is made from a normal Merca- 
tor projection by rotating the cylinder (the developable 
surface) 90Xso that it lies tangent along a meridian (line 
of longitude) instead of line of latitude. 


The Universal Transverse Mercator (UTM) coordinate 
system is based on a cartesian coordinate system. It is 
composed of 60 zones, each 6X of longitude wide, with 
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Universal Transverse Mercator (UTM) Coordinate System 
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FIGURE 7-13 There are 60 UTM zones, each 6Xof longitude wide from zone 01 at 180XW of the Prime Meridian to zone 60 
at 180M E. Each UTM zone has both a north and a south zone. The north zone extends from the Equator to 84XN latitude. The 
south zone extends from 80S latitude to the Equator (NGA, 2007) (MODIS imagery courtesy of NASA Goddard Space Flight 


Center; Reto StWckli). 


a Central Meridian placed every sixth meridian begin- 
ning with 177% West longitude. Zone 1 is located from 
180X to 174K West longitude. Zone 2 extends from 174X 
west to 168X West longitude and so forth. Each UTM 
zone has both a north and a south zone (Figure 7-13a). 
The north zone extends from the Equator to 84XN lat- 
itude. The southern zone extends from 84XS latitude to 
the Equator (Figure 7-13 and 7-14a). The UTM map 
projection is not used to map the poles. 


The intersection of the Central Meridian of a UTM 
and the Equator is the origin of the zone (Figure 7- 
14a). It has a central scale factor of 0.9996. The Cen- 
tral Meridians, the Equator, and each line 90X from the 
Central Meridian are straight lines (Figure 7-14a). The 
Central Meridian normally has a constant scale. Any 
lines parallel to the Central Meridian are lines of con- 
stant scale. UTM maps can be edge-matched only if 
they are in the same zone with one Central Meridian. 


Each UTM zone has a Central Meridian that remains 
perpendicular to the zone’s southern boundary 
throughout the north-south extent of the zone. Each 
zone also has a false-easting of 500,000 m west of the 
zone’s Central Meridian. This ensures that all easting 
and northing values remain positive throughout the 
zone. Otherwise, you would have negative values west 
of the Central Meridian. 


The UTM projection is often used for mapping large 
areas that are mainly north-south in extent. For exam- 
ple, the UTM projection is often used by the U.S. Geo- 
logical Survey in its topographic mapping program at 
scales from 1:24,000 to 1:250,000. UTM Zone 17 N is 
useful for mapping South Carolina with a Central Me- 
ridian of 81K W (Figure 7-14b). UTM Zone 16 N is 
useful for mapping Alabama and is centered on 87M W 
longitude (Figure 7-14b). 


Distances are true only along the Central Meridian se- 
lected by the image analyst or along two lines parallel 
to it, but all distances, directions, shapes, and areas are 
reasonably accurate within 15X of the Central Meridi- 
an. Distortion of distances, directions, and size of areas 
increase rapidly outside the 15X band. Because the map 
is conformal, however, shapes and angles within any 
small area (such as that found on a 1:24,000-scale 
USGS topographic map) are essentially true. 


UTM coordinates are in meters and referenced accord- 
ing to the x (easting) and y (northing) values. Easting 
refers to the meters east of the artificial zone boundary. 
Northing refers to meters north of the artificial zone 
boundary. In the northern hemisphere, northing refers 
to meters north of the Equator. For example, suppose 
that you are given the following UTM coordinate in- 
formation: 
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Characteristics 
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FIGURE 7-14 Universal Transverse Mercator (UTM) grid zone with associated parameters. This projection is often used when 
rectifying remote sensor data to a base map. It is found on U.S. Geological Survey 7.5- and 15-minute quadrangles. a) Charac- 
teristics of a UTM Grid Zone. There are 60 zones, each 6Min size encompassing the Earth. b) UTM grid Zone 17 N with a Cen- 
tral Meridian at 814 W longitude is useful for geospatial projects in South Carolina. UTM Zone 16 N centered on 87K W 
longitude is useful for projects in Alabama. Elevation data courtesy of the U.S. Geological Survey. 


12N 444782 E 4455672 N 


This means that the location is in Zone 12 north of the 
Equator, 444,782 meters east of the UTM origin and 
4,455,672 meters north of the Equator. Of course, you 
can also convert this location to other units. For exam- 
ple, you can divide these values by 1,000 to convert the 
units to kilometers. In this case, you could determine 
that the location is 444.782 km east of the UTM origin 
and 4,455.672 km north of the Equator. 


UTM coordinates are easy to use and virtually all GPS 
devices allow for locations to be mapped in UTM co- 
ordinates. In addition, the UTM coordinate system 
covers most of the world (except for the poles). Also, 
because the system uses meters as its base unit, it is 
easy to convert to other units and to calculate useful 
areal units such as meters~, hectares, and kilometers”. 
Internet mapping applications, such as Google Earth, 
allow for coordinates to be displayed and queried in 
UTM. 


Some disadvantages to the UTM coordinate system 
must be noted. It is often difficult to use a UTM map 
projection when the study area crosses multiple UTM 
zones from east to west. Therefore, if your study area is 
located across two UTM zones, a UTM map projec- 


tion may not be the best choice. This makes using 
UTM very difficult for large, continental-sized areas 
such as all of the conterminous United States or the 
entire area of the Amazon Basin. 


Space Oblique Mercator Projection: The useful 
Space Oblique Mercator (SOM) projection is a modi- 
fied cylindrical projection with the map surface defined 
by a satellite orbit. It was developed by U.S. Geological 
Survey scientists in the 1970s to reduce the amount of 
distortion caused when satellite images of the ellipsoi- 
dal Earth were printed on a flat page. It was originally 
developed to map the imagery obtained by the Landsat 
Multispectral Scanner (MSS). The projection is used 
to map the continuous swath of remote sensing data 
collected by a satellite during each orbital pass. The 
scale is true along the ground track (Snyder, 1987; 
1995). 


The SOM projection is useful mainly for the relatively 
narrow swath along the groundtrack of the satellite. 
SOM maps are basically conformal with the extent of 
the map defined by the orbital characteristics of the re- 
mote sensing system. The SOM may be used for any 
satellite orbiting the Earth in a circular or elliptical or- 
bit and at any inclination. 
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Perspective Map Projections 
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FIGURE 7-15 Perspective map projections: Gnomonic, Stereographic, and Orthographic. Note the location of the light 
source in relation to the location of the transparent globe and the developable surface, which in this particular example is a 
plane. Also note the angle of the shadows cast on the developable surface (Jensen and Jensen, 2013). 


Azimuthal (Planar) Map Projections 

Azimuthal map projections are mathematically pro- 
jected onto a plane tangent to (touching) any point on 
the globe, such as at the North Pole (90XN, 0X W) (Fig- 
ure 7-11c). They can also be created by intersecting the 
sphere with a plane, e.g., along 40M N (Figure 7-11d). 
The patterns of distortion associated with azimuthal 
map projections that just touch (tangent to) or inter- 
sect (secant to) the globe are shown in Figure 7-1 lcd 
(Slocum et al., 2008; Krygier, 2011). Azimuthal map 
projections may be centered on the poles (referred to as 
Polar-aspect), at a point on the Equator (referred to as 
Equatorial-aspect), or at any other orientation desired 
(referred to as Oblique-aspect). 


Perspective Azimuthal Map Projections: Perspec- 
tive Azimuthal map projections may be constructed 
geometrically using a light source placed at a certain 
location within or outside of a transparent globe that 
contains parallels of latitude and meridians of longi- 
tude (Figure 7-15). Rays of light from the light source 
project shadows of the meridians and parallels onto 
the planar surface. 


The location of the light source can be inside the 
sphere (referred to as gnomonic), on the opposite side 
of the sphere (the antipode) at the point of tangency 
(referred to as stereographic), or at an infinite distance 
from the point of tangency, yielding parallel light rays 
(referred to as orthographic) (Figure 7-15). In reality, 
the graticule (grid) of the globe is projected mathemati- 
cally onto the plane rather than by using light project- 
ed through the sphere. 


The Azimuthal Gnomonic projection is believed to be 
the oldest projection, developed around the sixth cen- 
tury B.C. Its most important property is that the Equa- 
tor and all meridians are mapped as straight lines, 


making it possible to find the shortest route between 
any two points. 


Azimuthal Stereographic maps are often used to dis- 
play the hemisphere that is opposite the light source 
point. It is not possible to show both hemispheres in 
their entirety. It is the only azimuthal projection that 
preserves true angles and local shape. It is often used to 
map large continent-sized areas of similar extent in all 
directions (USGS, 2011). 


The Azimuthal Orthographic projection is often used 
to display the Earth, moon, and other planets as if the 
viewer were infinitely far away in outer space. The dis- 
play closely resembles a three-dimensional view of the 
spheres. It is the most familiar of the azimuthal map 
projections (Intergraph, 2013). 


Lambert Azimuthal Equal-area: The Lambert Azi- 
muthal Equal-area map projection (Figure 7-16) is best 
suited for regions extending equally in all directions 
from a standard point of tangency. It is possible to lo- 
cate the point of tangency at the poles (e.g., Figure 7- 
16a) or anywhere else the user desires such as at 40XN, 
OX W longitude (Figure 7-16b) or at 40X N, 80% W lon- 
gitude as demonstrated using MODIS imagery of the 
Earth (Figure 7-16c). 


Areas on the map are shown in their true proportion to 
the same areas on the Earth. Therefore, any two quad- 
rangles that are bounded by two parallels and two me- 
ridians at the same latitude are equal in area. 
Directions are true only from the standard point of 
tangency. The scale decreases (becomes smaller) gradu- 
ally away from the standard point of tangency. The dis- 
tortion of shapes increases away from the standard 
point of tangency. Any straight line drawn through the 
standard point is on a great circle. The map is equal- 


Azimuthal Map Projections 
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b. Lambert Azimuthal Equal-area 40° N, 0° W. 


c. Lambert Azimuthal Equal-area 40° N, 80° W. 


FIGURE 7-16 Selected Lambert Azimuthal Equal-area map projections. a) Lambert Azimuthal Equal-area map projection of 
MOD S satellite imager y w th the standard poi nt of tangency at 90XN, OXW. b) Lambert Azimuthal Equal-area map projection 
of MODIS data with the point of tangency at 40XN, OX W. c) The point of tangency is at 40X N, 80 W (based on the use of 


NASA map projection software and NASA MODIS imagery). 


area but not conformal, perspective, or equidistant 
(USGS, 2011). 


Azimuthal Equidistant: In an Azimuthal Equidis- 
tant map projection, distances and directions to all 
places are true only from the standard point of tangen- 
cy of the projection. Distances are correct between 
points along straight lines through the standard (cen- 
ter) point. All other distances are inaccurate. Any 
straight line drawn through the standard (center) point 
is on a great circle. The distortion of areas and shapes 
increases away from the standard point (USGS, 2011). 
Azimuthal Equidistant projections are useful for show- 
ing airline distances from the standard point of the 
projection. A polar aspect is often used for world maps 
and maps of polar hemispheres. An oblique aspect is 


used for atlas maps of continents and world maps for 
aviation use. 


Conical Map Projections 

Conical map projections are mathematically projected 
onto a cone-developable surface. The cone can be tan- 
gent or secant to the sphere. If it is tangent, there is one 
standard parallel. If it is secant, there are two standard 
parallels (Figure 7-1lef). The greater the distance from 
the point of tangency or intersection of the sphere with 
the cone, the greater the distortion (Slocum et al., 
2008; Krygier, 2011). 


Albers Equal-area Conic: The Albers Equal-area 
Conic map projection is used by the U.S. Geological 
Survey for maps showing the conterminous United 
States (i.e., the 48 mainland states) or for mapping 
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Conical Map Projections 


a. Albers Equal-area Conic. 


b. Lambert Conformal Conic. 


FIGURE 7-17 Selected conical map projections. a) Albers Equal-area Conic map projection. b) Lambert Conformal Conic 
map projection (based on the use of NASA map projection software and NASA MODIS imagery). 


large areas of the United States (Figure 7-17a). The 
map projection is secant at two standard parallels. The 
projection is ideal when mapping large areas that are 
mainly east-west in geographic extent and when it is 
important to maintain equal-area relationships. The 
projection is not conformal, perspective, or equidis- 
tant. Directional measurement is reasonably accurate 
within limited geographic regions. Distance measure- 
ments are accurate along both of the standard paral- 
lels. The scale is true only along the two standard 
parallels. Maps created using the Albers Equal-area 
Conic projection can be edge-matched only if they 
have the same standard parallels and the same scale 
(USGS, 2011). 


Lambert Conformal Conic: The Lambert Confor- 
mal Conic map projection (Figure 7-17b) is one of the 
most widely used map projections. It is secant at two 
standard parallels. It looks like the Albers Equal-area 
Conic (Figure 7-17a), but the graticule spacing is dif- 
ferent. The projection is used by the USGS for many 


7.5- and 15-minute topographic maps and for the State 
Base Map series. It is also used to map countries or re- 
gions that are mainly east-west in extent (USGS, 2011). 


The Lambert Conformal Conic projection is not per- 
spective, equal-area, or equidistant. Distance measure- 
ment is true only along the two standard parallels and 
reasonably accurate elsewhere in limited regions. Di- 
rectional measurements are reasonably accurate, espe- 
cially near the standard parallels. The distortion of 
shapes and areas is minimized along the two standard 
parallels selected by the map maker, but increases the 
greater the distance away from the standard parallels. 
The shapes of areas on large-scale maps of relatively 
small geographic areas are essentially true. 


The two standard parallels are 33M N and 45XN for the 
USGS Base Map series for the 48 conterminous States. 
The standard parallels are varied for the very impor- 
tant USGS Topographic Map series (both 7.5- and 15- 
minute) (USGS, 2011). 


Utah State Plane Coordinate System 


FIGURE 7-18 Each state has its own State Plane Coordinate 
System. The size and shape of each state determines the 

number of zones and their orientation. Utah has three paral- 
lel State Plane Coordinate zones (North, Central, and South). 


Selected map projections ideally suited to mapping dif- 
ferent geographic areas are summarized in Table 7-4. 
The globe is best at portraying the world. Azimuthal 
projections are especially useful for mapping entire 
hemispheres, continents, and regions. The cylindrical 
Transverse Mercator and several conic map projections 
are especially useful for medium- and large-scale map- 
ping projects. 


Other Projections and Coordinate Systems 
Useful for Image Rectification 


Sometimes map projections or coordinate systems are 
specially prepared for political or other considerations. 
For example, in the 1930s each of the states agreed to 
develop their own map projection and coordinate sys- 
tem based on either the Transverse Mercator or the 
Lambert Conformal Conic map projections. These 
projections and coordinate systems came to be known 
as the State Plane Coordinate Systems (SPCS). 


State Plane Coordinate Systems: In the U.S., indi- 
vidual state projections were chosen to decrease distor- 
tion and minimize error. These projections were 
chosen based on an individual state’s shape and geo- 
graphic location on the sphere. In addition, some states 
decided to break up their State Plane Coordinate Sys- 
tem into different zones. For example, Utah’s State 
Plane Coordinate System contains three different 
zones (North, Central, and South) based on the state’s 
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geographic shape and location (Figure 7-18). Utah’s 
State Plane Coordinate System uses a Lambert Con- 
formal Conic projection. 


Most local government thematic data layers are based 
on the use of State Plane Coordinate Systems. Most le- 
gal descriptions used by governments include details in 
State Plane Coordinate System coordinates. So, if you 
plan on working for a local city or government agency 
you should be prepared to use the State Plane Coordi- 
nate System. 


As with UTM, there are disadvantages to working with 
the State Plane Coordinate System. Similar to UTM, 
the State Plane Coordinate System used in most states 
has multiple zones that are difficult to move across if 
your study area happens to straddle the zones. In addi- 
tion, State Plane Coordinate Systems end at the state 
line. Therefore, any projects that wish to examine mul- 
tiple states will need to select another coordinate sys- 
tem. The original unit of measurement for the State 
Plane Coordinate System was based on the NAD27 
datum in feet. Recently, the U.S. Geological Survey be- 
gan releasing metric topographic maps. These maps are 
based on a new State Plane Coordinate System that is 
based on the NAD83 datum, with meters as the mea- 
surement unit. 


Ground Control Point Collection 


Twenty ground control points were located on the map 
and the UTM easting and northing of each point were 
identified (Table 7-5). The same 20 GCPs were then 
identified in the TM data according to their row and 
column coordinates (Table 7-5). The location of points 
13, 14, and 16 is shown in Figure 7-5. The GCPs 
should be located uniformly throughout the region to 
be rectified and not congested into a small area simply 
because a) there are more easily identifiable points in 
that area or b) the locations are easy to get to in the 
field. 


Determine Optimum Geometric 
Rectification Coefficients by 
Evaluating GCP Total RMS.,,o,; 


The 20 GCPs selected were input to the least-squares 
regression procedure previously discussed to identify 1) 
the coefficients of the coordinate transformation, and 
2) the individual and total RMS,,;9, associated with 
the G@ s. A threshold of 0.5 was not satisfied until 13 
GCPs were deleted from the analysis. The order in 
which the 13 GCPs were deleted and the RMS.,;o; to- 
tal found after each deletion are summarized in Table 
7-5. The seven GCPs finally selected that produced an 
acceptable RMS,,,5; are shown in Table 7-6. GCP 11 
would have been the next point deleted if the threshold 
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TABLE 7-5 Characteristics of 20 ground control points used to rectify the Charleston, SC, Landsat Thematic Mapper scene. 


Order of Total RMSerror 
Points Easting on Northing on after This Point 
Point Number Deleted® Map, X, Map, X2 X’ Pixel Y' Pixel Is Deleted 

1 12 597,120 3,627,050 150 185 0.501 

2 9 597,680 3,627,800 166 165 0.663 

3 Kept 598,285 3,627,280 191 180 — 

4 Kept 595,650 3,627,730 98 179 — 

5 2 596,750 3,625,600 123 252 6.569 

6 13 597,830 3,624,820 192 294 0.435 

7 Kept 596,250 3,624,380 137 293 — 

8 Kept 602,200 3,628,530 318 115 — 

9 Kept 600,350 3,629,730 248 83 — 
10 5 600,680 3,629,340 259 93 1.291 
4 Kept 600,440 3,628,860 255 113 — 
12 10 599,150 3,626,990 221 186 0.601 
13 8 600,300 3,626,030 266 211 0.742 
14 6 598,840 3,626,460 211 205 1.113 
15 3 598,940 3,623,430 214 295 4.773 
16 Kept 600,540 3,626,450 272 196 — 
AZ 4 596,985 3,629,350 134 123 1.950 
18 7 596,035 3,627,880 109 174 0.881 
19 11 600,995 3,630,000 269 71 0.566 
20 1 601,700 3,632,580 283 12 8.542 

Total RMSerror With all 20 GCPs used: 11.016 


a.For example, GCP 20 was the first point deleted. After it was deleted, the total RMS.,;o; dropped from 11.016 to 8.542. Point 5 was the 
second point deleted. After it was deleted, the total RMS,,,9, dropped from 8.542 to 6.569. 


had not been satisfied. The six coefficients derived 
from the seven suitable GCPs are found in Table 7-7. 
The Charleston Landsat TM scene rectified using these 
parameters is shown in Figure 7-6a. 


It is instructive to demonstrate how the six coefficients 
were computed for this Landsat TM dataset. This was 
accomplished using only the final seven GCPs, as these 
represent the set that produced an acceptable RMS,,,6; 
of <0.50 pixels. Remember, however, that this same op- 
eration was performed with 20 points, then 19 points, 
and so on, before arriving at just 7 acceptable points. 


A least-squares multiple regression approach is used to 
compute the coefficients. Two equations are necessary. 
One equation computes the image y’ coordinate (the 


dependent variable) as a function of the x and y map 
coordinates (representing two independent variables). 
The second computes the image x’ coordinate as a 
function of the same map (x, y) coordinates. Three co- 
efficients are determined by using each algorithm. The 
mathematics for computing the column coordinates 
(x') in the image can now be presented. 


Multiple Regression Coefficients Computation 
First, we will let 


Y = either the x’ or y’ location in the image, 
depending on which is being evaluated; in 
this example it will represent the x’ values 

X = easting coordinate (x) of the map GCP 
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TABLE 7-6 Information concerning the final seven ground control points used to rectify the Charleston, SC, Landsat Themat- 


ic Mapper Scene. 


Easting on Adjusted Northing on Adjusted a 
Point Number Map Easting, X,? Map Northing, X2° X' Pixel Y' Pixel 
3 598,285 2,635 3,627,280 2,900 191 180 
4 595,650 0 3,627,730 3,350 98 179 
7 596,250 600 3,624,380 0 137 293 
8 602,200 6,550 3,628,530 4,150 318 115 
9 600,350 4,700 3,629,730 5,350 248 83 
11 600,440 4,790 3,628,860 4,480 255 113 
16 600,540 4,890 3,626,450 2,070 272 196 
Minimum = 24,165 Minimum = 22,300 1,519 1,159 
595,650 3,624,380 


a.Adjusted easting values (X1) used in the least-squares computation of coefficients. This is an independent variable. 
b.Adjusted northing values (X>) used in the least-squares computation of coefficients. This is an independent variable. 
c.The dependent variable (Y) discussed in the text. In this example it was used to predict the X’ pixel location. 


TABLE 7-7 Coefficients used to rectify the Charleston, SC, 
Landsat Thematic Mapper scene. 


I. Find (¥ 7 ) and (X¥ cy ) in deviation form: 


x' = -382.2366 + 0.034187x + (-0.005481)y 


y’ = 130,162 + (-0.005576)x + (-0.0349150)y 


where x, y are coordinates in the output image and x’, y’ 
are predicted image coordinates in the original, unrectified 
image. 


X, = northing coordinate (y) of the map GCP 


It is practical here to use X; and XY} instead of x and y 
to simplify mathematical notation. 


The seven coordinates used in the computation of the 
coefficients are shown in Table 7-6. Notice that the in- 
dependent variables (X; and X>) have been adjusted 
value = original value— minimum value) so that the 
sums of squares or sums of products do not become so 
large that they overwhelm the precision of the CPU be- 
ing used. Note, for example, that most of the original 
northing UTM measurements associated with the map 
GCPs are already in the range of 3 million meters. The 
minimums subtracted in Table 7-6 are added back into 
the analysis at the final stage of coefficient computa- 
tion. Next we will discuss the mathematics necessary to 
compute the x’ coefficients shown in Table 7-7. It is 
technical, but should be of value to those interested in 
how to compute the coefficients used in Equation 7.9. 


n= 7, the number of control points used 


A. First compute: 


vy, = 1,519 4 = 24,165 


i=1 t=1 


ou, = 22,300 


= 1 


Y=217 X, = 3,452.1428 


ae = 366,491 


t=1 


Xy = 3,185.7142 


xt = 119,151,925 YX = 89,832,800 


t=1 i=1 


DAY; = 6,385,519 X,Y, = 5,234,140 


i=1 i=1 


> Xo: = 91,550,500 


i=1 
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B. Compute sums of squares: 


n n 2 
1 1 
a i, = 2 2 
3 X}, » m 119,151,925 — >(24,165) 


= 35,730,892.8571 


n n % 
yh - 5 = 89,832,800 — + (22,300)? 


18,791,371.4286 


SrtA Sea | $5 
r=1 i=1 i=1 


91,550,500 — +(24,165)(22,300) 


14,567,714.2857 
where 


(XTX) = 35,730,892.8571 Nor attanee| 
14,567,714.2857 18,791,371.4286 


4. Covariance between Y and X): 


seo en 


i=1 i=] i=1 


= 6,385,515 — = (24,165)(1,519) 


1,141,710 
5. Covariance between Y and X: 


sor bial 


P= 1 a) i=1 


5,234,140 — = (22,300)(1,519) 


= 395,040 
where 


1,141,710 
a 395,040 


II. Find the inverse of (XY Sg y= (XxX ix 7: 


A. First, find the determinant of the 2 2 matrix: 


IX7X| = (35,730,892.8571)(18,791,371.4286) 
—(14,567,714.2857)? 
= 459,214,179,643,488.9 


B. Determine adjoint matrix of (X”Y) where adjoint 
equals the transpose of the cofactor matrix: 


18,791,371.4286 —14,567,714.2857 
Adjoint™ = 


—14,567,714.2857 35,730,892.8571 
*Note that if 


- 1 d—b 
A= ab then 4 | = } 
. d det A|_¢ q 
C Geen - by multiplying the adjoint of (X7X) 
by the det (X° X ) under 1. 


TTS AES) 
TY) = 
ce) Ceornneer 


18,791,373.7142 —14,567,714.2857 

—14,567,714.2857 35,730,896.4285 

- {oAne oe 
-0.32x10" 0.78x10" 


III. Now, compute coefficients using 


a, = (X"x) (XY): 


041x107 0.321077) . Eee : 
0.32 x 107 0.78 x10” 395,040 


a, = (0.000000041 (1,141,710) + (—0.000000032)(395,040) 
ay = (-0.000000032)( 1,141,710) + (—0.000000078)(395,040) 


a, = 0.0341877 


a, = —0.0054810 


Now compute the intercept, ao, from 


a) = Y- 5 ak 
i=l 


(The minimums of X; and X> [595,650 and 3,624,380, 
respectively] must be accounted for here. Please refer to 
Table 7-6.) 


dy = 217 — [(0.0341877)(3,452.1428 + 595,650) + 
(—0.005481)(3,185.7142 + 3,624,380)] = —382.2366479. 
Therefore, the equation becomes 


Y = -382.2366479 + 0.0341877X, — 0.00548 10X5. 


Because we actually evaluated the dependent variable 
x', this becomes 


x’ = —382.2366479 + 0.0341877x — 0.00548 10y 


with x and y representing the map coordinates and x’ 
being the predicted column coordinate in the original 
input image. 


Similar procedures are required to compute the other 
three coefficients for the row (y’ ) location in the input 
image. This would require inserting the seven y’ pixel 
values in the equation in Table 7-6 instead of the x’ 
pixel values just used. The X, and X> values associated 
with the x, y coordinates of the map remain the same. 


Fill Output Matrix Using Spatial and 
Intensity Interpolation Resampling 


With the coefficients computed, it was then necessary 
to 1) identify the UTM coordinates of the area on the 
map to be rectified, 2) select the type of intensity inter- 
polation to be performed (e.g., nearest-neighbor, bilin- 
ear, or cubic convolution), and 3) specify the desired 
output pixel size. A nearest-neighbor resampling algo- 
rithm was selected with a desired output pixel dimen- 
sion grid of 30 X 30 m. The finer the dimension of the 
output grid, the greater the number of computations 
required to fill it. Normally, the size of the pixel is 
made square (e.g., 30 X 30 m) to facilitate scaling con- 
siderations when the rectified data are displayed on 
computer monitors and various hard-copy output de- 
vices. These procedures resulted in rectified Landsat 
Thematic Mapper data. 


Mosaicking 


Mosaicking is the process of combining multiple imag- 
es into a single seamless composite image. It is possible 
to mosaic unrectified individual frames or flightlines of 
remotely sensed data. However, it is more common to 
mosaic multiple images that have already been rectified 
to a standard map projection and datum (Figure 7-19). 


Mosaicking Rectified Images 


Mosaicking n rectified images requires several steps. 
First, the individual images should be rectified to the 
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Mosaic Feathering Logic 


User-specified feathering 
distance, e.g., 200 pixels 


Base image 


a. Cut-line feathering. 


User-specified 
feathering 
distance, 
e.g., 200 pixels 


Base image 


b. Edge feathering. 


FIGURE 7-19 The visual seam between adjacent images 
being mosaicked may be minimized using a) cut-line feather- 
ing logic, or b) edge feathering. 


same map projection and datum. Ideally, rectification 
of the n images is performed using the same intensity 
interpolation resampling logic (e.g., nearest-neighbor) 
and pixel size (e.g., multiple Landsat TM scenes to be 
mosaicked are often resampled to 30 X 30 m). 


Next, one of the images to be mosaicked is designated 
as the base image. The base image and image 2 will 
normally overlap a certain amount (e.g., 20% — 30%). 
A representative geographic area in the overlap region 
is identified. This area in the base image is contrast 
stretched according to user specifications. The histo- 
gram of this geographic area in the base image is ex- 
tracted. The histogram from the base image is then 
applied to image 2 using a histogram-matching algo- 
rithm. This causes the two images to have approxi- 
mately the same brightness value characteristics. 


It is possible to have the pixel brightness values in one 
scene simply dominate the pixel values in the overlap- 
ping scene. Unfortunately, this can result in noticeable 
seams in the final mosaic. Therefore, it is common to 
blend the seams between mosaicked images using 
feathering (Tucker et al., 2004). Some digital image 
processing systems allow the user to specific a feather- 
ing buffer distance (e.g., 200 pixels) wherein 0% of the 
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Image Mosaicking 


~~ Strom Thurmond—» 
Reservoir 


base image 


a. Rectified Landsat ETM* image of eastern Georgia 
obtained on October 3, 2001 (bands 4, 3, 2; 
Worldwide Reference System—Path 18, Row 37). 


Savannah 
River Site 


b. Rectified Landsat ETMt image of western South 
Carolina obtained on October 26, 2001 (bands 4, 3, 2; 
Worldwide Reference System—Path 17, Row 37). 


Augusta, GA—- 


c. Feathered mosaic of rectified Landsat ETM* imagery of eastern Georgia and western South Carolina. 


FIGURE 7- 
Original imagery courtesy of NASA. 


base image is used in the blending at the edge and 
100% of image 2 is used to make the output image 
(e.g., Figure 7-19a). At the specified distance (e.g., 200 
pixels) in from the edge, 100% of the base image is used 
to make the output image and 0% of image 2 is used. 
At 100 pixels in from the edge, 50% of each image is 
used to make the output file. 


Sometimes analysts prefer to use a linear feature such 
as a river, road, or agricultural field boundaries to sub- 
due the edge between adjacent mosaicked images. In 
this case, the analyst identifies a polyline in the image 
(using an annotation tool) and then specifies a buffer 
distance away from the line as before where the feather- 
ing will take place (Figure 7-19b). It is not absolutely 


) Two Landsat Enhanced Thematic Mapper Plus (ETM*) images are mosaicked using feathering logic. 


necessary to use natural or human-made features when 
performing cut-line feathering. Any user-specified 
polyline will do. The output file is then produced. It 
consists of two histogram-matched images that have 
been feathered at the common edge(s) of the dataset. 
Hopefully, the multiple image edge-match is not visi- 
ble. Additional images (3, 4, 5, etc.) to be mosaicked 
are histogram-matched and feathered using similar 
logic. 


The output file of 7 mosaicked images should appear 
as if it were one continuous image. For example, Figure 
7-20a—b depicts two rectified Landsat Enhanced The- 
matic Mapper Plus (ETM“) scenes obtained over east- 
ern Georgia and western South Carolina on October 3, 
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2011 Mosaic of LANDSAT EMT* Imagery of the Conterminous United States 


a. Annual. 


b. Annual and seasonal. 


WELD: 
Web-enabled 
Landsat Data 


F RE 7-21 a) 2011 annual mosaic of Landsat ETM* image data. ETM* imagery was provided by the USGS (http:// 

weld. cr.usgs.gov/region_ds.php). Digital image processing was performed by the NASA-sponsored WELD: Web-enabled 
Landsat Data project (Roy et al., 2010; 2011; USGS WELD, 2014). b) Annual and seasonal periods of interest. c) Monthly peri- 
ods of interest. d) Weekly periods of interest. WELD interface and imagery courtesy of the U.S. Geological Survey and NASA. 


2001, and October 26, 2001, respectively. Each image 
was resampled using nearest-neighbor logic to 30 30 
m pixels in a UTM projection. The two Landsat 
ETM” frames contained approximately 20% side-lap. 
Note the Strom Thurmond Reservoir in the overlap re- 
gion. The two images were histogram-matched and 


mosaicked using edge-feathering logic (Figure 7-20c). 
This is a good mosaic because it is difficult to detect 
the seam between the two input images. 


Mosaicked images that span entire counties, states, and 
countries are especially important for many biological 
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Mosaic of Three Rectified NASA ATLAS Multispectral Flightlines of Myrtle Beach, SC 


Intercoastal 
Waterway 


Airport 


Myrtle Beach 


2 Mosaic of three NASA ATLAS 1 1 m multispectral flightlines of Myrtle Beach, SC, collected on October 7, 


1997 (original imagery courtesy of NASA). 


and geophysical investigations. For this reason, numer- 
ous agencies and data centers routinely provide mo- 
saicked products for scientific research. For example, 
the NASA-funded Web-enabled Landsat Data 
(WELD) project systematically generated 30 X 30 m 
composite Landsat Enhanced Thematic Mapper Plus 
(ETM*) mosaics of the conterminous United States 
and Alaska from 2002 to 2012 (USGS WELD, 2014). 
A much-reduced Landsat ETM™ mosaic of the conter- 
minous United States in 2011 is shown in Figure 7-21a. 
It is composed of hundreds of ETM™ images acquired 
during 2011. The mosaicking methodology is de- 
scribed in Roy et al. (2010; 2011). Unlike most mo- 
saicking projects, WELD does not just select cloud- 


free scenes to be included in the mosaic. Instead, it an- 
alyzes a number of scenes in the annual/seasonal, 
monthly, or weekly time period under investigation 
and uses special criteria to create a composite mosaic 
on a pixel-by-pixel basis. The criteria (e.g., per pixel 
saturation, cloudiness, NDVI <0.5, maximal NDVI) 
are specified in the WELD Algorithm Theoretical Basis 
Document (Roy et al., 2011). The mosaic output is in 
an Albers Equal Area Projection, which is ideal for 
east-west extending areas such as the conterminous 
United States (Snyder, 1995). 


Mosaicking multiple flightlines of aircraft panchro- 
matic, multispectral, or hyperspectral is usually more 


difficult than rectifying satellite imagery. A mosaic of 
three flightlines of NASA ATLAS 1% 1 m multispec- 
tral data is shown in Figure 7-22. The color balance be- 
tween the three flightlines is reasonable but there is 
some noticeable geometric error at the common edges 
of the flightlines. Additional carefully selected ground 
control points and perhaps a higher order polynomial 
could improve the mosaic. 


Conclusion 


Toutin (2004) posed the question “Why does the geo- 
metric correction process seem to be more important 
today than before?” He suggests that in 1972, the im- 
portance of image geometric distortion was negligible 
because: 


¢ Landsat MSS-like images were acquired at Nadir at 
relatively coarse spatial resolution, typically > 80 X 
80 m; 

¢ the images were in hard-copy paper or positive 
transparency format and analyzed visually; and 


¢ there was very little fusion of multiple types of geo- 
spatial information taking place. 


Today, geometric distortion is much more of a concern 
because: 


* the spatial resolution of the remote sensor data is 
much finer (e.g., often < 1X 1m) 


* many images are acquired at off-Nadir viewing 
angles [Chiu et al., 2011)); 

* the image data and the remote sensing—derived out- 
put products are digital (Devaraj and Shah, 2014); 

* it is common to fuse or merge different types of 
remote sensor data to complete a project (e.g., 
Green, 2011); and 

* itis common to merge both raster and vector digital 
geospatial data for modeling purposes. 


Geometrically accurate remote sensor data are re- 
quired to create remote sensing-derived products that 
the general public can trust. 
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8 IMAGE ENHANCEMENT 


Enhancement algorithms are applied to remotely 
sensed data to improve the appearance of an image for 
human visual analysis or occasionally for subsequent 
machine analysis. There is no such thing as the ideal or 
best image enhancement because the results are ulti- 
mately evaluated by humans, who make subjective 
judgments as to whether an enhanced image is useful. 


ey Overview 


This chapter identifies a variety of digital image en- 
hancement operations that have proven of value for vi- 
sual analysis of remote sensor data and/or subsequent 
machine analysis. Point operations modify the bright- 
ness values of each pixel in an image dataset indepen- 
dent of the characteristics of neighboring pixels. Local 
operations modify the value of each pixel in the context 
of the brightness values of the pixels surrounding it. 
Various types of image enhancement using either point 
or local operations are introduced including: image re- 
duction and magnification, the collection of informa- 
tion along a transect (a spatial profile), contrast 
stretching, density slicing, spatial filtering, Fourier 
analysis, vegetation transforms, texture mapping, and 
the derivation of several landscape ecology metrics. 


| Image Reduction and 
Magnification 


Image analysts routinely view images that have been re- 
duced in size or magnified during the image interpreta- 
tion process. Image reduction techniques allow the 
analyst to obtain a regional perspective of the remotely 


Source: NASA 


sensed data. Image magnification techniques allow the 
analyst to zoom in and view very site-specific pixel 
characteristics. 


Image Reduction 


In the early stages of a remote sensing project it is often 
necessary to view the entire image in order to locate the 
row and column coordinates of a subimage that en- 
compasses the study area. Most commercially available 
remote sensor data are composed of more than 3,000 
rows X 3,000 columns and a number of bands. Unfor- 
tunately, most digital image processing system displays 
cannot display this many pixels at one time. Therefore, 
it is useful to have a simple procedure for reducing the 
size of the original image dataset down to a smaller da- 
taset that can be viewed on the screen at one time for 
orientation purposes. To reduce a digital image to just 
1/m? of the original data, every mth row and mth col- 
umn of the imagery are systematically selected and dis- 
played. For example, consider the Advanced 
Spaceborne Thermal Emission and Reflection Radi- 
ometer (ASTER) image of Oahu, HI, that originally 
had 4,104 rows x 3,638 columns shown in Figure 8-1. 
The size of the original image was reduced by using ev- 
ery other row and every other column (i.e., m = 2) in 
the dataset. The sampled image shown consists of just 
2,052 rows X 1,819 columns. This reduced dataset con- 
tains only one fourth (25%) of the pixels found in the 
original scene. The logic associated with a simple 2x 
integer reduction is shown in Figure 8-2a. 


If you were to compare the original ASTER data with 
the reduced ASTER data, there is an obvious loss of 
detail because so many of the pixels are not present. 
Therefore, we rarely apply digital image processing 
techniques to image reductions. Instead, they are used 
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2x Reduction of NASA ASTER Image of Oahu, HI, 
Obtained on June 3, 2000 (RGB = Bands 2, 3, 1) 


Original columns = 3,638 @ 30 m; Displayed columns = 1,819 
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FIGURE 8-1 This 2,052 row& 1,819 column image represents only 25% of the data found in the original 4,104 rowX 3,638 
column NASA Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) image. It was created by sam- 
pling every other row and column using the logic shown in Figure 8-2 (courtesy NASA/GSFC/METI/ERSDAC/JAROS and U.S./ 


Japan ASTER Science Team). 


for orienting within a scene and locating the row and 
column coordinates of specific areas-of-interest (AOT) 
that can then be extracted at full resolution for image 
analysis. 


Image Magnification 


Digital image magnification (often referred to as zoom- 
ing in) is usually performed to enlarge the scale of an 
image or map for visual interpretation or, occasionally, 
to match the scale of another image or map. Just as 
row and column deletion is the simplest form of image 
or map reduction, row and column replication repre- 
sents the simplest form of digital magnification. To 
magnify a digital image or map by an integer factor 
mx, each pixel in the original image or map is usually 
replaced by an m X m block of pixels, all with the same 
value as the original input pixel. The logic of a 2x mag- 


nification is shown in Figure 8-2b. This form of magni- 
fication is characterized by visible square tiles of pixels 
in the output display. Image magnifications of 1x, 2x, 
and 3x applied to the ASTER image of Oahu, HI, are 
shown in Figure 8-3. 


Most sophisticated digital image processing systems al- 
low an analyst to specify floating-point magnification 
(or reduction) factors (e.g., zoom in 2.75x). This re- 
quires that the original remote sensor or map data be 
resampled in near real-time using one of the standard 
resampling algorithms (e.g., nearest-neighbor, bilinear 
interpolation, or cubic convolution discussed in Chap- 
ter 7). This is a very useful technique when the analyst 
is trying to obtain detailed information about the char- 
acteristics of a small geographic area of interest. Dur- 
ing the training phase of a supervised classification (to 
be discussed in Chapter 9), it is especially useful to be 
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Local Reduction and Magnification Operations Applied to A Single Raster Dataset 


Integer 2x Reduction 
Input Raster 


Output Raster 
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Integer 2x Magnification 


_ Input Raster Output Raster 


FIGURE 8-2 a) The logic associated with 24 image integer reduction using local pixel operations that sample every other row 
and column of the original dataset. This operation results in a new image consisting of only one quarter (25%) of the original 


data. b) The logic of 2X image magnification. 


able to zoom in to the raw remote sensor data at very 
precise floating point increments to isolate a particular 
field or body of water. 


In addition to magnification, virtually all digital image 
processing systems provide a mechanism whereby the 
analyst can pan or roam about a much larger geograph- 
ic area (e.g., 2,048 < 2,048) while viewing only a por- 
tion (e.g., 512 X 512) of this area at any one time. This 
allows the analyst to view parts of the database much 
more rapidly. 


Transects (Spatial Profiles) 


The ability to extract brightness values along a user- 
specified transect (also referred to as a spatial profile) 
between two or multiple points in a single-band or 
multiple-band color composite image is important in 
many remote sensing image interpretation applica- 
tions. For example, consider the three simple 50-pixel 
horizontal spatial profiles (A, B, and C) identified in 
the single-band, black-and-white thermal infrared im- 
age of the Savannah River (Figure 8-4a). In each case, 
the spatial profile in histogram format depicts the mag- 
nitude of the brightness value at each pixel along the 
50-pixel transect. Each pixel in the thermal image was 
2.8 X 2.8 m. 


Sometimes it is useful to view the spatial profile histo- 
gram in a single grayscale tone (or color) as shown in 
Figure 8-4b. Conversely, analysts often prefer to dis- 


play the grayscale tone of the individual brightness val- 
ues encountered along the transect as shown in Figures 
8-4c, d, and e. 


Each of the transects in Figure 8-4 was located along a 
single horizontal scan line. In this case, each of the 
transects was exactly 140 m long (50 pixels x 2.8 m = 
140 m). But what if we wanted to extract a detailed 
transect across the river perpendicular to the shoreline? 
Or what if we wanted to run a transect through the 
heart of the plume to see how fast the thermal plume 
was cooling after entering the river? In this case it 
would be necessary to take into account the stair- 
stepped nature of the transect and incorporate distance 
information derived using the Pythagorean theorem 
previously discussed. Examples of a 7-pixel diagonal 
transect and a 7-pixel horizontal transect are shown in 
Figure 8-5. 


Accurate transect results can also be obtained by rotat- 
ing the image until the desired transect is aligned with 
either a single line or column in the dataset. For exam- 
ple, consider the density-sliced image of the Savannah 
River thermal plume shown in Figure 8-6b based on 
the color look-up table values in Table 8-1. In this ex- 
ample, we are interested in identifying a) the tempera- 
ture of the thermal plume in the river perpendicular to 
the shoreline using a 50-pixel transect (1), and b) the 
rate of temperature decrease as the plume progresses 
downstream using a 100-pixel transect (2). The bright- 
ness values encountered along the two transects were 
extracted only after the original image was geometri- 
cally rotated 16M in the clockwise direction so that the 
endpoints of each transect fell on the same scan line or 
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Magnification of ASTER Imagery of Oahu, HI 


a. 1x magnification 
(original resolution). 


b. 2x magnification. 
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FIGURE 8-3 NASA ASTER imagery of Oahu, HI, magnified 14, 2X, and 3X (courtesy NASA/GSFC/METI/ERSDAC/JAROS and 
U.S./Japan ASTER Science Team). 
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FIGURE 8-4 a) Three 50-pixel spatial profiles 
(transects) passed through the Savannah Riv- 
er predawn thermal infrared data. Each pixel 
in the dataset is 2.8 K 2.8 m. b) The spatial 
profile data are displayed in histogram for- 
mat using a single grayscale tone. c-e) The 
spatial profile data are displayed in histogr am 
format according to their original brightness 
values. 


Brightness Value 


column (Figure 8-6c). This ensured that the number of 
meters in each temperature class along each transect 
was accurately measured. If the analyst extracts tran- 
sects where the endpoints do not fall on the same scan 
line (or column), the hypotenuse of stair-stepped pixels 
must be considered instead of the simple horizontal 
pixel distance (Figure 8-5). 
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Distance in Pixels 


Transects 1 and 2 are shown in Figure 8-6d and e. The 
relationship between the original brightness values and 
the class intervals of the transects is provided in Table 
8-1. By counting the number of pixels along a transect 
in specific temperature-class intervals within the plume 
and counting the total number of pixels of river (Table 
8-1), it is possible to determine the proportion of the 
thermal plume falling within specific temperature-class 
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Transect Logic 


Ending 
pixel in 
transect 


Beginning pixel 
in transect 


a. Transect on a diagonal. a 


b. Transect along a 
row (or column). 


19.6m 


FIGURE 8-5 a) The stair-stepped nature of a transect (spatial profile) when the beginning and ending points are not on the 
same line (or column). In this example, a 45X angle transect consisting of just seven 2.8% 2.8 m pixels would be 27.72 m in 
length. b) A 7-pixel transect along a row (or column) of the image would be only 19.6 m long. Therefore, when diagonal tran- 
sects (or spatial profiles) are extracted, it is necessary to compute their length based on the Pythagorean theorem. 


TABLE 8-1 Savannah River thermal plume density slice specifications used to create Figure 8-6bc. The number of pixels in 
each class interval associated with Transect 1 is provided. 


Relationship of Class to Ambient River Temperature 


Class 2 Class 7 
Light blue White 
(RGB = 0, 0, (RGB = 255, 
255) 255, 255) 
mis >20°C 
Transect 1° | Width of Brightness Value Range for Each Class Interval 
River 
177-255 
50 pixels | 39 pixels 4/11.2 


@2.8m | =109.2m 


a. Transect | was 140 m long (50 pixels at 2.8 m/pixel). Transect measurements in the river were made only after the image 
was rotated so that the beginning and ending pixels of the transect fell on the same scan line. 

b. Includes | mixed pixel of land and water on each side of the river. 

c. Notation represents pixels and meters; for example, 24 pixels represent 67.2 m. 


intervals (Jensen et al., 1983; 1986). In 1981, the South more than one third of the width of the river. Transect 
Carolina Department of Health and Environmental 1 information extracted from thermal infrared imagery 
Control (DHEC) mandated that a thermal plume (summarized in Table 8-1) can be used to determine if 
could not be >2.8Mabove river ambient temperature for — the plume was in compliance at that location. 
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Transects (Spatial Profiles) 
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FIGURE 8-6 Transects (spatial profiles) passed through a predawn thermal infrared image of a thermal plume located in the 
Savannah River. a) Original image. b) Original image density-sliced according to the logic presented in Table 8-1. c) Density- 
sliced image rotated 16M with Transects 1 and 2 displayed. d) Spatial profile of Transect 1. e) Spatial profile of Transect 2. 


Spectral Profiles 


In addition to extracting pixel brightness values or per- 
cent reflectance values along selected spatial profiles 
(transects), it is often useful to extract the full spec- 
trum of brightness values in bands for an individual 
pixel. This is commonly referred to as a spectral profile. 


In a spectral profile, the x-axis identifies the number of 
the individual bands in the dataset and the y-axis docu- 
ments the brightness value (or percent reflectance if the 
data have been calibrated) of the pixel under investiga- 
tion for each of the bands (Figure 8-7e). 


The usefulness of the spectral profile is dependent 
upon the quality of information in the spectral data. 
Analysts sometimes assume that in order to do quality 
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Spectral Profiles Extracted from SPOT 20 x 20 m Data 


a. Band | (Green; 0.5 — 0.59 um). 


b. Band 2 (Red; 0.61 — 0.68 um). 


c. Band 3 (Near-infrared; 0.79 — 0.89 um). 
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FIGURE 8-7 a-c) Three bands of SPOT 20 20 m multispectral data of Marco Island, FL. d) A color composite of SPOT bands 
3, 2, and 1. e) Spectral profiles of mangrove, beach sand, and ocean water extracted from the multispectral data (SPOT imag- 


ery courtesy of Airbus Defense and Space). 


remote sensing research they need a tremendous num- 
ber of bands. Sometimes this is true. However, at other 
times just two or three optimally located bands in the 
electromagnetic spectrum can be sufficient to extract 
the desired information and solve a problem. There- 
fore, the goal is to have just the right number of opti- 
mally located, non-redundant spectral bands. Spectral 
profiles can assist the analyst by providing unique visu- 
al and quantitative information about the spectral 


characteristics of the phenomena under investigation 
and whether there are any serious problems with the 
spectral characteristics of the dataset. 


Spectral profiles extracted from 1) mangrove, 2) sand, 
and 3) water locations in a 20 < 20 m three-band SPOT 
image (green, red, and near-infrared) of Marco Island, 
FL, are shown in Figure 8-7e. Note that these data 
have not been converted into percent reflectance; 
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Spectral Profiles Extracted from HyMap Hyperspectral Data 
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6OK GR NTR: 


50 


40 


30 


Percent Reflectance 


= | 


4. “green” turf 


d. Color composite (RGB = bands 40, 15, 9). 


golf 


15 ot 
Wavelength, Xm 


FIGU RE 8-8 a-c) Three bands of HyMap hyperspectral data of the Debordiu colony near North Inlet, SC. The data were ob- 
tained at a spatial resolution of 3X 3 m. d) Color composite of HyMap bands 40, 15, and 9. e) Spectral profiles of golf “green” 
turf, sand, roof, and water extracted from the 116 bands of hyperspectral data. 


therefore, the y-axis is simply labeled Brightness Value. 
Because this dataset consists of only three bands, it is 
useful but not very informative. As expected, the pixel 
of mangrove under investigation (1) absorbs more red 
light than green light due to chlorophyll a absorption 
and reflects significant amounts of the incident near- 
infrared energy. The sandy beach (2) reflects approxi- 
mately equal amounts of green, red, and near-infrared 
energy. As expected, the green light is reflected slightly 
more than the red light for the water pixel (3) while 


most of the incident near-infrared radiant flux is ab- 
sorbed by the water, causing the brightness value to ap- 
proach zero. 


A second example demonstrates the information con- 
tent of hyperspectral remote sensing data. The spectral 
profiles for features located in HyMap hyperspectral 
data of the Debordiu residential colony near North In- 
let, SC, are shown in Figure 8-8. The hyperspectral da- 
taset consists of 116 atmospherically corrected bands 
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of data at 3 x 3 m spatial resolution. Please note that 
the data have been calibrated to percent reflectance 
units. Only 3 of the 116 bands are displayed, including 
band 9 (green), band 15 (red), and band 40 (near-infra- 
red). The spectral profile for the golf-putting green (1) 
exhibits all the characteristics of a well-calibrated hy- 
perspectral vegetation pixel. There is chlorophyll ab- 
sorption in the blue and red portions of the spectrum 
and significant reflectance throughout the near-infra- 
red part of the spectrum. Reflectance in the middle-in- 
frared bands in the region 1.55 — 1.75 Xm and 2.08 — 
2.35 Xm is also strong. The atmospheric water absorp- 
tion bands at 1.4 and 1.9 Xm are evident in all the spec- 
tra. The spectral profile for sand (2) is high throughout 
the visible bands (blue, green, and red), causing it to 
appear bright white to human observers. The spectral 
profile for a residential rooftop (3) reflects relatively 
lower amounts of blue, green, and red energy through- 
out the visible spectrum, causing it to appear gray. Fi- 
nally, water (4) absorbs more and more incident energy 
as we progress from the visible into the near-and mid- 
dle-infrared portions of the spectrum. 


Contrast Enhancement 


Remote sensing systems record reflected and emitted 
radiant flux exiting from Earth’s surface materials. Ide- 
ally, one material would reflect a tremendous amount 
of energy in a certain wavelength and another material 
would reflect much less energy in the same wavelength. 
This would result in contrast between the two types of 
material when recorded by the remote sensing system. 
Unfortunately, different materials often reflect similar 
amounts of radiant flux throughout the visible, near- 
infrared, and middle-infrared portions of the electro- 
magnetic spectrum, resulting in relatively /ow-contrast 
imagery. In addition, besides this obvious low-contrast 
characteristic of biophysical materials, there are cultur- 
al factors at work. For example, people in developing 
countries often construct urban areas using natural 
building materials (e.g., wood, sand, silt, clay). This 
can cause urbanized areas in developing countries to 
have about the same reflectance characteristics as the 
neighboring countryside. Conversely, urban infrastruc- 
ture in developed countries is usually composed of 
concrete, asphalt, and fertilized green vegetation. This 
typically causes urbanized areas in developed countries 
to have reflectance characteristics significantly differ- 
ent from the surrounding countryside. 


An additional factor in the creation of low-contrast re- 
motely sensed imagery is the sensitivity of the detec- 
tors. For example, the detectors on most remote 
sensing systems are designed to record a relatively wide 
range of scene brightness values (e.g., 0 to 255) without 


becoming saturated. Saturation occurs if the radiomet- 
ric sensitivity of a detector is insufficient to record the 
full range of intensities of reflected or emitted energy 
emanating from the scene. The Landsat TM detectors, 
for example, must be sensitive to reflectance from di- 
verse biophysical materials such as dark volcanic ba- 
salt outcrops or snow (possibly represented as BVs of 0 
and 255, respectively). However, very few scenes are 
composed of brightness values that use the full sensi- 
tivity range of the Landsat TM detectors. Therefore, 
this results in relatively low-contrast imagery, with 
original brightness values that often range from ap- 
proximately 0 to 100. 


To improve the contrast of digital remotely sensed da- 
ta, it is desirable to use the entire brightness range of 
the display or hard-copy output device (discussed in 
Chapter 5). Digital methods may be more useful than 
photographic techniques for contrast enhancement be- 
cause of the precision and wide variety of processes 
that can be applied to the imagery. There are linear and 
nonlinear digital contrast-enhancement techniques. 


Linear Contrast Enhancement 


Contrast enhancement (also referred to as contrast 
stretching) expands the original input brightness values 
to make use of the total dynamic range or sensitivity of 
the output device. To illustrate the linear contrast- 
stretching process, consider the Charleston, SC, TM 
band 4 image produced by a sensor system whose im- 
age output levels can vary from 0 to 255. A histogram 
of this image is provided (Figure 8-9a). We will assume 
that the output device (a high-resolution display) can 
display 256 shades of gray (1.e., quant, = 255). The his- 
togram and associated statistics of this band 4 subim- 
age reveal that the scene is composed of brightness 
values ranging from a minimum of 4 (1.e., ming = 4) to 
a maximum value of 105 (1.e., maxy = 105), with a 
mean of 27.3 and a standard deviation of 15.76 (refer 
to Table 4-7). When these data are displayed on the 
screen without any contrast enhancement, we use less 
than one-half of the full dynamic range of brightness 
values that could be displayed (1.e., brightness values 
between 0 and 3 and between 106 and 255 are not 
used). The image is rather dark, low in contrast, with 
no distinctive bright areas (Figure 8-9a). It is difficult 
to visually interpret such an image. A more useful dis- 
play can be produced if we expand the range of origi- 
nal brightness values to use the full dynamic range of 
the video display. 


Minimum—Maximum Contrast Stretch 

Linear contrast enhancement is best applied to remote- 
ly sensed images with Gaussian or near-Gaussian his- 
tograms, that is, when all the brightness values fall 
generally within a single, relatively narrow range of the 
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Contrast Stretching of Landsat TM Band 4 Image of Charleston, SC 
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FIGURE 8-9 a) Original Landsat Thematic Mapper band 4 data of Charleston, SC, and its histogram. This image has not been 
contrast stretched. b) Minimum-—maximum contrast stretch applied to the data and the resultant histogram. c) One standard 
deviation (- 16) linear contrast stretch applied to the data and the resultant histogram. d) Application of histogram equaliza- 


tion and the resultant histogram. Original imagery courtesy of NASA. 


histogram and only one mode is apparent. Unfortu- spectively. The output brightness value, BV,,;, is 


nately, this is rarely the case, especially for scenes that | computed according to the equation: 
contain both land and water bodies. To perform a lin- 

ear contrast enhancement, the analyst examines the BV..,— min, 

image statistics and determines the minimum and max- BY 5 = | | quant,., 
: * ‘ ‘ max, — min 
imum brightness values in band k, min; and max,, re- k k 


(8.1) 
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FIGURE 8-10 a) The result of applying a minimum—maximum contrast stretch to normally distributed remotely sensed data. 
The histograms before and after the transformation are shown. The minimum and maximum brightness values encountered in 
band k are min, and max,, respectively. b) Theoretical result of applying a—1 standard deviation percentage linear contrast 
stretch. This moves the min, and max; values — 34% from the mean into the tails of the distribution. 


where BV;,, is the original input brightness value and 
quant, is the maximum value of the range of brightness 
values that can be displayed on the CRT (e.g., 255). In 
the Charleston, SC, example, any pixel with a BV;,,, of 4 
would now have a BV,,, of 0, and any pixel with a BV;, 
of 105 would have a BV,, of 255: 


_ =) 
BV out ey 255 


BV yy, = 0 

Mey 
BV out cee 255 
BV, = 255. 


The original brightness values between 5 and 104 
would be linearly distributed between 0 and 255, re- 
spectively. The application of this enhancement to the 
Charleston TM band 4 data is shown in Figure 8-9b. 
This is commonly referred to as a minimum—maximum 
contrast stretch. Most image processing systems pro- 


vide for the display of a before-and-after histogram, as 
well as a graph of the relationship between the input 
brightness value (BV;,,) and the output brightness value 
(BV5y). For example, the histogram of the min-max 
contrast stretch discussed is shown in Figure 8-9b. 


The logic of a min-max linear contrast stretch is shown 
diagrammatically in Figure 8-10a. Note the linear rela- 
tionship between the brightness values of the input and 
output pixels and how the slope of the line would be- 
come increasingly steep as the minimum is increased or 
the maximum is decreased. The application of a mini- 
mum—maximum contrast stretch enhancement to the 
Savannah River predawn thermal infrared data is 
shown in Figure 8-11b. 


Percentage Linear and Standard Deviation 
Contrast Stretching 


Image analysts often specify min, and max, that lie a 
certain percentage of pixels from the mean of the histo- 
gram. This is called a percentage linear contrast stretch. 
If the percentage selected coincides with a standard de- 
viation percentage, then it is called a standard deviation 
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Contrast Stretching of Thermal Infrared Image of the Savannah River 
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FIG RE 8-11 a) Original predawn thermal infrared data of the Savannah River, not contrast stretched, and its histogram. b) 
Minimum-—maximum contrast stretch applied to the data and the resultant histogram. c) One standard deviation (— 16) per- 
centage linear contrast stretch applied to the data and resultant histogram. d) Application of histogram equalization and re- 


sultant histogram. 


contrast stretch. For normal distributions, 68% of the 
observations lie within —1 standard deviation of the 
mean, 95.4% of all observations lie within —2 standard 
deviations, and 99.73% within —3 standard deviations. 
Consider applying a +1 standard deviation contrast 
stretch to the Charleston, SC, Landsat TM band 4 da- 
ta. This would result in min, = 12 and max, = 43. All 


values between 12 and 43 would be linearly contrast 
stretched to lie within the range 0 to 255. All values be- 
tween 0 and 11 are now 0, and those between 44 and 
255 are set to 255. This results in more pure black-and- 
white pixels in the Charleston, SC, scene, dramatically 
increasing the contrast of the image, as shown in Fig- 
ure 8-9c. The information content of the pixels that 
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Piecewise Linear Contrast Stretching 


255 


BV out 


0 BV; n 255 


FIGURE 8-12 Logic of a piecewise linear contrast stretch 
for which three selective pieces of the histogram are linearly 
contrast stretched. Note that the slope of the linear con- 
trast enhancement changes. 


saturated at 0 and 255 is lost. The slope of a percentage 
linear or standard deviation contrast stretch is greater 
than for a simple min—max contrast stretch (refer to 
Figure 8-10b). 


The results of applying a —1 standard deviation linear 
contrast stretch to the thermal plume data are shown 
in Figure 8-11c along with the histogram. The — 1 stan- 
dard deviation contrast stretch effectively “burns out” 
the thermal plume, yet provides more detail about the 
temperature characteristics of vegetation on each side 
of the river. 


Piecewise Linear Contrast Stretch 


When the histogram of an image is not Gaussian (i.e., 
it is bimodal, trimodal, etc.), it is possible to apply a 
piecewise linear contrast stretch to the imagery of the 
type shown in Figure 8-12. Here the analyst identifies a 
number of linear enhancement steps that expand the 
brightness ranges in the modes of the histogram. In ef- 
fect, this corresponds to setting up a series of min; and 
max, and using Equation 8.1 within user-selected re- 
gions of the histogram. This powerful contrast en- 
hancement method should be used when the analyst is 
intimately familiar with the various modes of the histo- 
gram and what they represent in the real world. Such 
contrast-stretched data are rarely used in subsequent 
image classification. 


To perform piecewise linear contrast enhancement, the 
analyst normally views a) the raw image and b) a dis- 
play of the histogram of the raw image superimposed 
with an input-output line that runs diagonally from 
the lower left to the upper right in the display. The ana- 
lyst then interactively adjusts the length and slope of 
mutually exclusive contrast stretches along the input— 
output line. 


To illustrate the process, consider the two piecewise lin- 
ear contrast enhancements found in Figure 8-13. The 
Savannah River thermal plume is composed primarily 
of values from 81 to 170. A special contrast stretch to 
highlight just the plume is demonstrated in Figures 8- 
13ab (note the red lines). Upland and ambient Savan- 
nah River water brightness values from 0 to 80 are sent 
to 255 (white), values from 81 to 170 are linearly con- 
trast stretched to have values from 0 to 255, and all val- 
ues from 171 to 255 now have a value of 255 (white). 
Conversely, if we wanted to contrast stretch the image 
to highlight the spectral characteristics of the ambient 
Savannah River, we might use the logic shown in Fig- 
ures 8-13cd. In this example, the spike in the original 
histogram associated with just the ambient Savannah 
River water has been singled out and linearly contrast 
stretched to have values from 0 to 255. All the land and 
thermal plume pixels are set to a value of 0 and appear 
black. 


Nonlinear Contrast Enhancement 


Nonlinear contrast enhancements may also be applied. 
One of the most useful enhancements is histogram 
equalization. The algorithm passes through the individ- 
ual bands of the dataset and assigns approximately an 
equal number of pixels to each of the user-specified 
output grayscale classes (e.g., 32, 64, 256). Histogram 
equalization applies the greatest contrast enhancement 
to the most populated range of brightness values in the 
image. It automatically reduces the contrast in the very 
light or dark parts of the image associated with the 
tails of a normally distributed histogram. 


Histogram equalization is found in many image pro- 
cessing systems because it requires very little informa- 
tion from the analyst to implement (usually just the 
number of output brightness value classes desired and 
the bands to be equalized), yet it is often very effective. 
Because of its wide availability, it is instructive to re- 
view how the equalization takes place using a hypo- 
thetical dataset (Gonzalez and Wintz, 1977). For 
example, consider an image that is composed of 64 
rows and 64 columns (4,096 pixels) with the range of 
brightness values that each pixel can assume, quant;, 
limited to 0 through 7 (Table 8-2). A histogram of this 
hypothetical image is shown in Figure 8-14a and the 
frequency of occurrence of the individual brightness 
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Piecewise Linear Contrast Stretching 
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d. Enhancing the Savannah River. 


FIGURE 8-13 a,b) Thermal infrared imagery of the Savannah River enhanced using piecewise linear contrast stretching to 
highlight the thermal plume at the expense of the ambient river wa ter and the surrounding landscape. c,d) Piecewise linear 
contrast stretching to enhance the Savannah River at the expense of the thermal plume and the surrounding landscape. 


values, f(BV;), is summarized in Table 8-2. For exam- 
ple, there are 790 pixels in the scene with a brightness 
value of 0 (1.e., f[BVo] = 790) and 1,023 pixels with a 
brightness value of 1 (ie, f[BV;] = 1,023). We can 
compute the probability of the ith brightness value, p,, 
by dividing each of the frequencies, f(BV;), by the total 
number of pixels in the scene (1.e., 7 = 4,096). Thus, the 
probability of encountering a pixel with a brightness 


value of 0 in the scene is approximately 19% (i.e., pp = 
ST [BV /n = 790/4,096 = 0.19). A plot of the probability 
of occurrence of each of the eight brightness values for 
the hypothetical scene is shown in Figure 8-14b. This 
particular histogram has a large number of pixels with 
low brightness values (0 and 1), making it a relatively 
low-contrast scene. 


288 INTRODUCTORY DIGITAL IMAGE PROCESSING 


TABLE 8-2 Statistics for a 64M 64 hypothetical image with 
Brightness Values from 0 to 7 (n = 4,096 pixels) (modified 
from Gonzalez and Wintz, 1977). 


Brightness 
Value, Frequency Probability 
BV; L f (BV) p; = f(BV;)/n 

BVo 0/7 = 0.00 790 0.19 
BV, 1/7 = 0.14 1023 0.25 
BV, 2/7 = 0.28 850 0.21 
BV3 3/7 = 0.42 656 0.16 
BV, 4/7 = 0.57 329 0.08 
BV, 5/7 = 0.71 245 0.06 
BV, 6/7 = 0.85 122 0.03 
BV, 7/7 = 1.00 81 0.02 


The next step in the process is to compute a transfor- 
mation function k; for each brightness value. One way 
to conceptualize the histogram equalization process is 
to use the notation shown in Table 8-3. For each 
brightness value level BV; in the quant, range of 0 to 7 
of the original histogram, a new cumulative frequency 
value k; is calculated: 

quant ap V,) 


n 


k. = 


1 


(8.2) 
i=0 


where the summation counts the frequency of pixels in 
the image with brightness values equal to or less than 
BV;, and n is the total number of pixels in the entire 
scene (4,096 in this example). The histogram equaliza- 
tion process iteratively compares the transformation 
function k; with the original values of L; to determine 
which are closest in value. The closest match is reas- 
signed to the appropriate brightness value. For exam- 
ple, in Table 8-3 we see that kg = 0.19 is closest to Ly = 
0.14. Therefore, all pixels in BVo (790 of them) will be 
assigned to BV,. Similarly, the 1,023 pixels in BV, will 
be assigned to BV3, the 850 pixels in BV, will be as- 
signed to BVs, the 656 pixels in BV3 will be assigned to 
BYV~,, the 329 pixels in BV, will also be assigned to BV¢, 
and all 448 brightness values in BVs5_7 will be assigned 
to BV;. The new image will have no pixels with bright- 
ness values of 0, 2, or 4. This is evident when evaluat- 
ing the new histogram (Figure 8-14d). When analysts 
see such gaps in image histograms, it is usually a good 
indication that histogram equalization or some other 
operation has been applied. 


Histogram-equalized versions of the Charleston TM 
band 4 data and the thermal plume data are found in 
Figure 8-9d and 8-11d, respectively. Histogram equal- 
ization is dramatically different from any other con- 
trast enhancement because the data are redistributed 
according to the cumulative frequency histogram of 
the data, as described. Note that after histogram equal- 
ization, some pixels that originally had different values 
are now assigned the same value (perhaps a loss of in- 
formation), while other values that were once very 
close together are now spread out, increasing the con- 
trast between them. Therefore, this enhancement may 
improve the visibility of detail in an image, but it also 
alters the relationship between brightness values and 
image structure (Russ, 2011). For these reasons, it is 
not wise to extract texture or biophysical information 
from imagery that has been histogram equalized. 


Another type of nonlinear contrast stretch involves 
scaling the input data logarithmically, as diagrammed 
in Figure 8-15. This enhancement has the greatest im- 
pact on the brightness values found in the darker part 
of the histogram. It could be reversed to enhance val- 
ues in the brighter part of the histogram by scaling the 
input data using an inverse log function, as shown. 


The selection of a contrast-enhancement algorithm de- 
pends on the nature of the original histogram and the 
elements of the scene that are of interest to the user. An 
experienced image analyst can usually identify an ap- 
propriate contrast-enhancement algorithm by examin- 
ing the image histogram and then experimenting until 
satisfactory results are obtained. Most contrast en- 
hancements cause some useful information to be lost. 
However, that which remains should be of value. Con- 
trast enhancement is applied primarily to improve visu- 
al image analysis. It is not good practice to contrast 
stretch the original imagery and then use the enhanced 
imagery for computer-assisted classification, change 
detection, etc. Contrast stretching can distort the origi- 
nal pixel values, often in a nonlinear fashion. 


Band Ratioing 


Sometimes differences in brightness values from identi- 
cal surface materials are caused by topographic slope 
and aspect, shadows, or seasonal changes in sunlight 
illumination angle and intensity. These conditions may 
hamper the ability of an interpreter or classification al- 
gorithm to identify correctly the surface materials or 
land use/land cover in a remotely sensed image. Fortu- 
nately, ratio transformations of the remotely sensed 
data can, in certain instances, be applied to reduce the 
effects of such environmental conditions. In addition 
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TABLE 8-3 Example of how a hypothetical 64 & 64 image with brightness values from 0 to 7 is histogram equalized. 


Frequency, f (BV;) 790 1023 850 656 329 245 122 81 
Original brightness value, 0 1 2 3 4 5 6 7 
i 
ee brightness value 0 0.14 0.28 0.42 0.57 0.71 0.85 1.0 
i n 
Cumulative frequency 790 1,813 2,663 3,319 3,648 3,893 4,015 4,096 
transformation: 4,096 4,096 4,096 4,096 4,096 4,096 4,096 4,096 
quant, f (BY;) =0.19 = 0.44 = 0.65 = 0.81 = 0.89 = 0.95 = 0.98 = 1.0 
k = a 
i=0 
Assign original BV; class to 1 3 5 6 6 7 7 7 
the new class it is 
closest to in value. 


Histogram Equalization Contrast Enhancement 
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FIGURE 8-14 Histogram equalization process applied to hypothetical data (based on Gonzalez and Wintz, 1977). a) Original 
histogram showing the frequency of pixels in each brightness value. b) Original histogram expressed in probabilities. c) The 
transformation function. d) The equalized histogram showing the frequency of pixels in each brightness value. 
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Linear and Nonlinear Contrast Stretching 
255 


BVout 


0 BVin 255 


FIGURE 8-15 Logjc of a linear, nonlinear logarithmic, and 
inverse logarithmic contrast stretch. 


to minimizing the effects of environmental factors, ra- 
tios may also provide unique information not available 
in any single band that is useful for discriminating be- 
tween soils and vegetation. 


The mathematical expression of the ratio function is: 


BV, 

a, — 8.3 
an By (8.3) 
where BV; ;,, is the output ratio value for the pixel at 
row i, column j and BV; ;, and BV; ;; are the brightness 
values at the same location in bands k and /, respective- 
ly. Unfortunately, the computation is not always simple 
because BV; ; = 0 is possible. However, there are alter- 
natives. For example, the mathematical domain of the 
function is 1/255 to 255 (i.e., the range of the ratio 
function includes all values beginning at 1/255, passing 
through 0, and ending at 255). The way to overcome 
this problem is simply to give any BV; ; with a value of 
0 the value of 1. Alternatively, some like to add a small 

value (e.g., 0.1) to the denominator if it equals zero. 


To represent the range of the function in a linear fash- 
ion and to encode the ratio values in a standard 8-bit 
format (values from 0 to 255), normalizing functions 
are applied. Using this normalizing function, the ratio 
value | is assigned the brightness value 128. Ratio val- 
ues within the range 1/255 to | are assigned values be- 
tween | and 128 by the function: 


BV; jn = Int [(BV;;, x 127) + 1]. (8.4) 


Ratio values from 1 to 255 are assigned values within 
the range 128 to 255 by the function: 


BY; 57 
BV 3 = Int (128 + bit) 


7 (8.5) 
Deciding which two bands to ratio is not always a sim- 
ple task. Often, the analyst simply displays various ra- 
tios and then selects the most visually appealing. The 
optimum index factor (OIF) and Sheffield Index (dis- 
cussed in Chapter 5) can be used to identify optimum 
bands for band ratioing (Chavez et al., 1984; Sheffield, 
1985). Crippen (1988) recommended that all data be 
atmospherically corrected and free from any sensor 
calibration problems (e.g., a detector is out of adjust- 
ment) before they are ratioed. 


Numerous band ratios of Landsat Thematic Mapper 
data of Charleston, SC, obtained on February 3, 1994, 
are shown in Figure 8-16. A color-composite consist- 
ing of bands 4, 3, and 2 is shown in Figure 8-16a. Band 
ratios of the visible bands (1 through 3) are shown in 
Figures 8-16b,c, and e. Not surprisingly, because the 
visible bands are so highly correlated, it is difficult to 
distinguish between some of the major land covers 
such as wetland versus water. Band ratios of the near- 
infrared band 4 data with the visible bands are shown 
in Figure 8-16d, f, g, and h. The 3/4 band ratio is the 
inverse of the 4/3 band ratio. The 4/3 near-infrared/red 
ratio provides vegetation information that will be dis- 
cussed in the section on vegetation indexes in this 
chapter. The brighter the pixel, the more vegetation or 
biomass present. Generally, the lower the correlation 
between the bands, the greater the information content 
of the band-ratioed image. For example, the ratio of 
bands 6 (thermal) and 4 (near-infrared) provides useful 
information (Figure 8-16k). Two color composites are 
displayed that incorporate three band-ratio images. 
For example, Figure 8-160 is a color composite of 
band ratios 1/4, 1/3, and 1/2 (RGB). Figure 8-16p is a 
color composite of band ratios 4/3, 1/3, and 6/4 (RGB). 
This band ratio is especially informative with water in 
light blue, wetland in light green, upland vegetation in 
yellow and red, and urbanized areas in shades of gray. 


As you would expect, there are a tremendous number 
of band ratio images that can be derived from a multi- 
spectral and especially a hyperspectral image dataset. 
The goal is to create ratios based on the knowledge of 
some physical process or principle such as the relation- 
ship that exists for healthy vegetation, 1.e., it always ab- 
sorbs most of the incident red light while reflecting 
most of the incident near-infrared light. Thus, a near- 
infrared/red ratio is ideal for identifying the presence of 
vegetation in an image as demonstrated. Other more 


a. Original RGB = bands 4,3,2. b. band 1 / band 2. 


n.7/4. 


m. 6/7. 


t Thematic Mapper I 
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magery of Charleston, SC, February 3, 1994 


o. RGB = 1/4, 1/3, 1/2. p. RGB = 4/3, 1/3, 6/4. 


FIGURE 8-16 The ratio of various Landsat TM bands of Charleston, SC, obtained on February 3, 1994. 


complex band ratio transformations are discussed in 
the vegetation index section later in this chapter. 


me Neighborhood Raster 
Operations 


Local raster operations such as image reduction, mag- 
nification, band ratioing, etc. modify the values of each 


pixel in an image independent of the characteristics of 
neighboring pixels. Conversely, neighborhood raster op- 
erations modify the value of each focal pixel in the con- 
text of the values of the pixels surrounding it. Some of 
the most commonly used raster neighborhood types 
are shown in Figure 8-17. The focal cell (f) is usually at 
the center of the neighborhood being examined, as 
shown in Figure 8-17a. 
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Convolution Masks of Various Shapes and Sizes 


FIGURE 8-17 Examples of various convolution masks. 


The raster rectangular neighborhood operators are 
probably the most heavily used, with 3 x 3 and 5 x 5 
being the most prevalent (Figure 8-17a). The 5 x 5 oc- 
tagonal, the 4 nearest-neighbor cross, and the edge- 
preserving median filter neighborhoods are also used 
when filtering images (Figure 8-17bc). Rectangular 
neighborhoods usually have an odd number of rows 
and columns so that there is always a central, focal pix- 
el that is being evaluated. It is important to remember, 
however, that an image analyst can specify virtually 
any size of rectangular neighborhood, e.g., 9 x 9, 17 x 
17, 25 x 25. 


The logic of using a rectangular neighborhood to per- 
form spatial filtering is shown in Figure 8-18. Note that 
in this example the 3 x 3 spatial moving window is 
stepped sequentially from left to right through the in- 
put raster dataset, line by line. Also note that the first 
row and column of the output matrix may not have any 
values when using a 3 X 3 neighborhood. In this case, 
the user must decide whether or not to leave the rows 
and columns blank or replicate the values from an ad- 
jacent row or column. 


4 Nearest- 
neighbor 


pal Edge-preserving (el at) 
ioe a median filter ha 


The circle neighborhood extends a user-specified radi- 
us away from the focal cell (Figure 8-17d). An annulus 
neighborhood centered on the focal cell includes the 
geographic area (i.e., pixels) bounded by the outer cir- 
cle and the inner circle (Figure 8-17e). Pixels within the 
innermost circle are not included in the annulus com- 
putation. The wedge neighborhood encompasses a 
piece of a circle radiating out from the focal cell (Fig- 
ure 8-17f). 


Note that often only a part of a given pixel (cell) is in- 
cluded within the boundaries of a circular, annulus, or 
wedge neighborhood. When this occurs, the pixels are 
included if their centroids (i.e., the exact center of the 
pixels) lie within the neighborhood. 


Qualitative Raster Neighborhood Modeling 

Some of the most important uses of raster neighbor- 
hood analysis involves nominal-scale data such as land 
cover (e.g., Class A, B, and C) or ordinal-scaled data 
(e.g., Good =1, Adequate =2, and Poor = 3). However, 
unlike the previous section which only involved the ex- 
amination of individual pixels in a single file or in mul- 
tiple registered files, raster neighborhood analysis 


Use of a Raster Rectangular Neighborhood 
to Perform Spatial Filtering 
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FIGURE 8-18 The logic of using a rectangular neighbor- 
hood to per form spatial filtering. Note that the spatial win- 
dow is stepped sequentially from left to right through the 
input raster, line by line. Also note that the first row and col- 
umn of the output matrix may not have any values when us- 
ing a 3M 3 neighborhood. In this case, the user must decide 
whether or not to leave them blank or replicate the values 
from an adjacent row or column. 


usually examines a focal cell as well as a predetermined 
number of pixels surrounding the focal cell and uses 
this information to assign a new value to the focal cell 
in a new output file. A few of the qualitative measures 
that can be determined are shown in Figure 8-19a. 


The application of a simple 3 x 3 window majority fil- 
ter is shown in Figure 8-19a. Majority filters are often 
used to remove the salt-and-pepper noise associated 
with remote sensing-derived land cover maps. The re- 
sult is a land cover thematic map that is much more 
pleasing to look at because an isolated pixel (e.g., wa- 
ter) that is totally surrounded by a majority of other 
types of land cover (e.g., forest) is assigned to the ma- 
jority land cover class that surrounds it. Sometimes it 
is good to determine the majority or minority of digital 
number values within a spatial moving window (Figure 
8-19a). 


Perhaps it is important to determine the diversity of 
values found within the spatial moving window (Figure 
8-19a). For example, a high diversity value for a pixel 
might indicate the presence of a large number of land 
cover types in a relatively small region (1.e., within the 3 
x 3 window). Such diversity might be especially impor- 
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tant for certain animals that require a diverse (hetero- 
geneous) land cover habitat for survival. Conversely, 
some animals might thrive best in homogeneous (non- 
diverse) land cover (e.g., dense monoculture forest 
stands). 


Quantitative Raster Neighborhood Modeling 
Raster neighborhood analysis is often used to analyze 
interval and ratio-scaled data. Several of the most 
common quantitative univariate descriptive statistic 
measurements extracted from n XM n spatial moving 
windows include the minimum, maximum, mean, and 
standard deviation as shown in Figure 8-19b. 


(Ry Spatial Filtering 


A characteristic of remotely sensed images is a parame- 
ter called spatial frequency, defined as the number of 
changes in brightness value per unit distance for any 
particular part of an image. If there are very few 
changes in brightness value over a given area in an im- 
age, this is commonly referred to as a low-frequency 
area. Conversely, if the brightness values change dra- 
matically over short distances, this is an area of high- 
frequency detail. Because spatial frequency by its very 
nature describes the brightness values over a spatial re- 
gion, it is necessary to adopt a spatial approach to ex- 
tracting quantitative spatial information. This is done 
by looking at the local (neighboring) pixel brightness 
values rather than just an independent pixel value. This 
perspective allows the analyst to extract useful spatial 
frequency information from the imagery. 


Spatial frequency in remotely sensed imagery may be 
enhanced or subdued using two different approaches. 
The first is spatial convolution filtering based primarily 
on the use of convolution masks. The procedure is rela- 
tively easy to understand and can be used to enhance 
low- and high-frequency detail, as well as edges in the 
imagery (Lo and Yeung, 2007). Another technique is 
Fourier analysis, which mathematically separates an 
image into its spatial frequency components. It is then 
possible interactively to emphasize certain groups (or 
bands) of frequencies relative to others and recombine 
the spatial frequencies to produce an enhanced image. 
We first introduce the technique of spatial convolution 
filtering and then proceed to the more mathematically 
challenging Fourier analysis. 


Spatial Convolution Filtering 


A linear spatial filter is a filter for which the value (V; ;) 
at location i, jin the output image or map is a function 
of some weighted average (linear combination) of the 
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Neighborhood Operations Applied to A Single Raster Dataset 
Using A 3 x 3 Spatial Moving Window 


Input Raster(s) 
Land Cover 
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Majority 3 
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1 = Wetland 
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Output Raster(s) 
Majority 


Number of Unique 
Values (Diversity) 


Minimum Mean 


Maximum Standard Deviation 


FIGURE 8-19 Neighborhood operations applied to a single raster dataset using a 3M 3 spatial moving window. a) Simple 


qualitative operations. b) Simple quantitative operations. 


values located in a particular spatial pattern around 
the 7, 7 location in the input image or map (Figure 8-18 
and 8-19). This process of evaluating the weighted 
neighboring pixel values is called two-dimensional con- 
volution filtering (Pratt, 2014). 


Sophisticated geographic information systems and re- 
mote sensing digital image processing software (e.g., 
ERDAS IMAGINE, ENVIJ) provide simple user inter- 
faces that allow the analyst to specify the size of the 
convolution kernel (e.g., 3 x 3, 5 x 5) and the coeffi- 
cients to be placed in the convolution kernel. Examples 
of ERDAS IMAGINE and ArcGIS ArcMap convolu- 
tion filtering user interfaces are shown in Figure 8-20. 


Spatial convolution filtering can be used to enhance 
low-frequency detail, high-frequency detail, and edges 
in raster digital images. The following discussion of 
spatial convolution filtering will be based on the appli- 
cation of various filters to a high-spatial-resolution 
color aerial photograph. Please remember, however, 
that the filters can also be applied to continuous raster 
thematic map data such as elevation, temperature, hu- 
midity, population density, etc. (Warner et al., 2009). 


Low-frequency Filtering in the Spatial Domain 
Raster enhancements that block or minimize the high- 
spatial frequency detail are called low-frequency or low- 
pass filters. The simplest low-frequency filter evaluates 
a particular input value, V;,,, and the pixels surround- 
ing the input pixel, and outputs a new value, V,,;, that 
is the mean of this convolution. The size of the neigh- 
borhood convolution mask or kernel (7) is often 3 x 3, 
5x 5,77, or 9 X 9. Examples of symmetric 3 x 3 and 
5 x 5 convolution masks are shown in Figure 8-17a. 
The following discussion will focus primarily on the 
use of 3 X 3 convolution masks with nine coefficients, 
c; defined at the following locations: 


Convolution mask template = | c 4 (8.6) 


C5 C6 
cy C8 Co 
For example, the coefficients in a low-frequency convo- 


lution mask are usually set equal to | (e.g., Figure 8- 
20ab and Table 8-4): 


Spatial Convolution Filtering User Interfaces 
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c. ArcGIS ArcMap filter interface. 
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Low-frequency filter = (8.7) 


The coefficients, c;, in the mask template are multi- 
plied by the following individual values (V;) in the in- 
put digital image: 


x V x V, x V. 


Cy AF yp Sos Foe Sa eg 


Mask template = cyx Vy egxVz cexV¢ (8.8) 


Cx Va cg x Ve CoXVo 


The primary input pixel under investigation at any one 
time is V5 = V;;. Convolution of the low-frequency fil- 
ter (with all coefficients equal to 1) and the original im- 
age data will result in a low-frequency filtered image 
where: 


LFF a (8.9) 


5,out ~ n 


Unsigned_8_bit 
{Output Unsigned_8_bit ~ 


Ignore Zero in Stats. 


! 957.00 


648.00 
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1.000 
1.000 


XSize: 3 2 Y Size: 3 > Default 0.00 


FIGURE 8-20 a) An example of the ERDAS 
IMAGINE convolution user interface. b) The ana- 
lyst can use the edit menu to input the desired 
coefficients in ann & nkernel. In this case, a3 W 3 


tesy of Hexagon Geospatial). c) Spatial filtering 
using the ArcGIS ArcMap spatial analyst filtering 
interface. A low-pass filter has been selected 
(interface courtesy of Esri, Inc.). 


low-pass filter is being prepared (interfaces cour- 


The spatial moving average then shifts to the next pix- 
el, where the average of all nine digital number values 
is computed. This operation is repeated for every pixel 
in the input image (Figure 8-18). Image smoothing is 
useful for removing “salt and pepper” noise in raster 
data. This simple smoothing operation will, however, 
blur the image, especially at the edges of objects. The 
blurring becomes more severe as the size of the kernel 
increases. 


A_ high spatial resolution (6 x 6 in.) normal color dig- 
ital aerial photograph of a residential area in Germany 
is shown in Figure 8-2la. It was obtained by an un- 
manned aerial vehicle (UAV) developed by senseFly, 
Inc. Application of a low-frequency filter (Equation 8.7 
and Table 8-4) to the red band of the residential image 
is shown in Figure 8-21b. Note how the image becomes 
blurred, suppressing the high-frequency detail. Only 
the general trends are allowed to pass through the low- 
pass filter. In a heterogeneous, high-frequency urban 
environment, a high-frequency filter usually provides 
superior results. 


Three spatial filtering algorithms (median, minimum/ 
maximum, and Olympic) do not use coefficients within 
the n K n neighborhood being examined. A median fil- 
ter has certain advantages when compared with 
weighted convolution filters (Russ, 2011), including 1) 
it does not shift boundaries, and 2) the minimal degra- 
dation to edges allows the median filter to be applied 
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TABLE 8-4 Selected low- and high-frequency filters and linear and nonlinear edge enhancement filters with their convolution 
mask coefficients. 


C1 Cy C3 1aid 
Convolution mask template = C4 Cs Ce Example: Low-frequency filter = |] 1 1 
Cr Cg Cg 111 
Cy | €n | €z | 4g | C5 | Cg | C7 | Cg | Co Example 
Spatial Filtering 
Low-frequency (LFF) 1 1 1 1 1 1 1 1 1 Figure 8-21b 
High-frequency (HFF) 1 |-2/; 1 );-2); 5); -2/ 1 |-2/ 1 Figure 8-21d 
Linear Edge Enhancement 
Emboss Northwest © | @]| 4 OF Om Om 1 iO 0 Figure 8-22a 
Emboss East 0;0;0) 1 Oo |} -1/ 0 0 0 --- 
ArcGIS Edge Enhancement -0.7| -1 |-0.7| -1 | 6.8 | -1 |-0.7| -1 |-0.7 --- 
Compass North 1 1 1 1 | -2]) 1 | -1 | -1 |} -1 --- 
Compass NE 1 1 1 |-1}-2) 1 | -1 | -1 1 Figure 8-22b 
Compass East -1 | 1 | al | 2 || a | 1 1 --- 
Compass SE -1|-1]) 1 |-1]-2] 1 1 1 1 --- 
Compass South -1 |-1]-1]} 1 ]-2] 1 1 1 1 
Compass SW 1 |-1/-1] 1 | -2] -1 1 1 1 
Compass West 1 1 | -1/] 1 | -2 |} -1 1 1 -1 
Compass NW 1 1 il 1 | -2 | -1 1 | -1/ -1 
Vertical Edges -1; 0] 1 /-1); 0 1 |-1/ 0 1 
Horizontal Edges -1|;-1/;-1; 0; 0), 0 1 1 1 
Diagonal Edges © | 4 ‘| | =i | © 1 | -1 | -1 0 
Nonlinear Edge Enhancement 
Laplacian 4 0 |;-1; 0 |-1] 4 | -1] O | -1 te) Figure 8-22c 
Laplacian 5 0 |;-1; 0 |-1/] 5 | -1] O | -1 0) Figure 8-22d 
Laplacian 7 1 1 1 1|-7/ 1 1 1 1 
Laplacian 8 -1 | -1 | -1 |} -1]) 8 | -1 | -1 | -1 | -1 
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Spatial Filtering of Raster Data 


a. Original contrast stretched. 


F he 


c. Median filter. 


d. High-frequency sharp edge filter applied to the red band. 


FIGURE 8-21 a) High spatial resolution color digital aerial phot ogr aphy of a resident ial area in Germany (original digital 
aerial photography courtesy of Sensefly, LLC). b) Low-frequency filter (LFF) applied to the red band. c) Median filter applied 
to the red, green, and blue bands. d) High-frequency sharp edge filter applied to the red band. 


repeatedly, which allows fine detail to be erased and 
large regions to take on the same brightness value (of- 
ten called posterization) (Figure 8-21c). The standard 
median filter will erase some lines in the image that are 
narrower than the half-width of the neighborhood and 
round or clip corners (Eliason and McEwen, 1990). An 
edge-preserving median filter (Nieminen et al., 1987) 
may be applied using the logic shown in Figure 8-17c, 
where 1) the median value of the black pixels is com- 
puted in a 5 X 5 array, 2) the median value of the gray 
pixels is computed, 3) these two values and the central 
original brightness value are ranked in ascending or- 
der, and 4) a final median value is selected to replace 
the central pixel. This filter generally preserves edges 
and corners. 


Minimum or maximum filters examine the digital num- 
ber values of adjacent pixels in a user-specified region 


(e.g., 3 X 3 pixels) and then replace the value of the 
center pixel with the minimum or maximum value en- 
countered. 


The Olympic filter is named after the system of scoring 
in Olympic events. Instead of using all nine elements in 
a 3 X 3 matrix, the highest and lowest values are 
dropped and the remaining values are averaged. 


High-frequency Filtering in the Spatial Domain 

High-pass filtering is applied to remotely-sensed data 
to remove the slowly varying components and enhance 
the high-frequency local variations. One useful high- 
frequency filter (HF Fs 5,1) is computed by subtracting 
the output of the low-frequency filter (LFF’s yt) from 
twice the value of the original central pixel value, BVs: 


AFF s oy = (2 X BVs) — LFF out- (8.10) 
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a. Emboss Northwest filter. 


c. Laplacian 4 filter. 


d. Laplacian 5 filter. 


FIGURE 8-22 Application of various convolution masks and logic to the red band of high spatial resolution imagery of a resi- 
dent ial area in Germany. a) Emboss Northwest filter. b) Compass Northeast directional filter. c) Laplacian 4 edge enhancement 
contains information about the edges. d) Laplacian 5 edge enhancement adds the edge data back onto the original image 


(based on original digital aerial photography from Sensefly, LLC). 


Brightness values tend to be highly correlated in a 
nine-element window. Thus, the high-frequency filtered 
image will have a relatively narrow intensity histogram. 
This suggests that the output from high-frequency fil- 
tered images often must be contrast stretched prior to 
visual analysis. High-frequency filtering is applied to 
raster maps or images to remove the slowly varying 
components and enhance the high-frequency local 
variations. One type of high-frequency filter that accen- 
tuates or sharpens edges uses the coefficients shown 
below and summarized in Table 8-4. 


1 2 1 
High-frequency filter = _2 5 _9 (8.11) 
1 2 1 


The application of this high-frequency sharp edge filter 
to the residential aerial photograph red band is shown 
in Figure 8-21d. 


Edge Enhancement in the Spatial Domain 
Often, the most valuable information that may be de- 
rived from an image or map is contained in the edges 
surrounding objects of interest. Edge enhancement de- 
lineates these edges in an image or map and makes 
them more conspicuous and often easier to under- 
stand. Generally, what the eyes see as pictorial edges 
are simply sharp changes in digital number value be- 
tween two adjacent pixels. The edges may be enhanced 
using either linear or nonlinear edge enhancement 
techniques. 


Linear Edge Enhancement Edge enhancement is 
often performed by convolving the original data with a 


weighted mask or kernel, as previously discussed. One 
of the most useful edge enhancements causes the edges 
to appear in a plastic shaded-relief format. This is of- 
ten referred to as embossing. Embossed edges may be 
obtained by using embossing filters such as the Em- 
boss East and Emboss Northwest shown below and in 
Table 8-4: 


oO 
oO 
oO 


Emboss East = {1 ( —] (8.12) 


oO 
oO 
oO 


00 1 
0 0 0 
-1 0 0 


Emboss NW 


(8.13) 


The direction of the embossing is controlled by chang- 
ing the value of the coefficients around the periphery 
of the mask. The plastic shaded-relief impression is 
pleasing to the human eye if shadows are made to fall 
toward the viewer. The red band of the residential im- 
age processed using the Emboss Northwest filter is 
shown in Figure 8-22a. 


The ArcGIS high-pass edge enhancement coefficients 
are provided in Table 8-4. This particular filter removes 
low-frequency variation and highlights the boundary 
(edges) between different regions. 


Compass gradient masks may be used to perform two- 
dimensional, discrete differentiation directional edge 
enhancement (Pratt, 2014). Table 8-4 lists the coeffi- 
cients used in eight commonly used compass gradient 
masks. The compass names suggest the slope direction 
of maximum response. For example, the east gradient 
mask produces a maximum output for horizontal digi- 
tal number value changes from west to east. The gradi- 
ent masks have zero weighting (i.e, the sum of the 
mask coefficients is zero) (Pratt, 2014). This results in 
no output response over regions with constant bright- 
ness values (i.e., no edges are present). A Compass 
Northeast gradient mask applied to the residential im- 
age is Shown in Figure 8-22b. 


Richards (2013) identified four additional 3 x 3 filters 
that may be used to detect edges in images (vertical, 
horizontal, and diagonal). These filter coefficients are 
listed in Table 8-4. 


Laplacian filters may be applied to imagery or continu- 
ous surface maps to perform edge enhancement. The 
Laplacian filter is a second derivative (as opposed to 
the gradient, which is a first derivative) and is invari- 
ant to rotation, meaning that it is insensitive to the di- 
rection in which the discontinuities (e.g., edges) run. 
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The coefficients associated with four important 3 x 3 
Laplacian filters are shown below and listed in Table 
8-4 (Jahne, 2005; Pratt, 2014): 


0-1 0 

Laplacian4= _, 4 _] (8.14) 
0-1 0 
0-1 0 

LaplacianS= _, 5 _| (8.15) 
0-1 0 
1 1 1 

Laplacian7= |_7 | (8.16) 
1 1 1 
-1-1-1 

Laplacian8= _,| 98 -] (8.17) 
—] -1 -1 


Sometimes by itself, the Laplacian image may be diffi- 
cult to interpret. For example, consider the Laplacian 4 
filter applied to the residential scene shown in Figure 
8-22c. Therefore, some analysts prefer using a Lapla- 
cian edge enhancement that adds the edge information 
back onto the original map or image using the Lapla- 
cian 5 algorithm. The result of applying this enhance- 
ment to the residential scene is shown is Figure 8-22d. 


The Laplacian 7 filter may be used to subtract the La- 
placian edges from the original image, if desired. Sub- 
tracting the Laplacian edge enhancement from the 
original image restores the overall grayscale variation, 
which the human viewer can comfortably interpret. It 
also sharpens the image by locally increasing the con- 
trast at discontinuities (Russ, 2011). The Laplacian op- 
erator generally highlights points, lines, and edges in 
the image and suppresses uniform and smoothly vary- 
ing regions. Human vision physiological research sug- 
gests that we see objects in much the same way. Hence, 
the use of this operation has a more natural look than 
many of the other edge-enhanced images. 


Nonlinear Edge Enhancement Nonlinear edge en- 
hancements are performed using nonlinear combina- 
tions of pixels. For example, the Sobel edge detector is 
based on the 3 x 3 window numbering scheme previ- 
ously described and is computed using the following 
equation: 


Sobel = /Xx2+Y2 (8.18) 


5,out 
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a. Sobel edge enhancement. 


b. Roberts edge enhancement. 


FIGUJ RE 8-23 Application of two nonlinear edge enhancements to the red band of the high spatial resolution residential im- 
agery. a) A Sobel edge enhancement. b) A Roberts edge enhancement (based on original digital aerial photography from 


Sensefly, LLC). 


where 


X = (Vtg +Vo)-(V, +2V 4+ Va) 


and 


Y= (V,+2V,+ V3)-(Vi+2Ve+Vo) 


This procedure detects horizontal, vertical, and diago- 
nal edges. A Sobel edge enhancement of the residential 
scene is found in Figure 8-23a. The Sobel operator may 
also be computed by simultaneously applying the fol- 
lowing 3 xX 3 templates across the image (Jain, 1989): 


-1 01 f 2 tf 
A= 2 ff 2 Y= 0 0 0 
ai), -O: a a, 23 <a 


The Roberts edge detector is based on the use of only 
four elements of a 3 X 3 mask. The new pixel value at 
pixel location V5 44; (refer to the 3 x 3 numbering 
scheme in Equation 8.6) is computed according to the 
equation: 


Robertss oy, = 4+ Y (8.19) 
where 
A= |5- V6| 
Y= \Ve= Ve| 


A Roberts edge filter is applied to the residential scene 
in Figure 8-23b. The Roberts’ operator also may be 
computed by simultaneously applying the following 
templates across the image (Jain, 1989): 


The Kir sch nonlinear edge enhancene nt calculates the 
gradient at pixel location BV; ;. To apply this operator, 
however, it is first necessary to designate a different 3 x 
3 window numbering scheme than used in previous dis- 
cussions: 


Window numbering for Kirsch = 
BV, BV, BV, 
BV, B Vii BV, 
BV, BV; BV 4 


The algorithm applied is (Gil et al., 1983): 


7 
BY; = max} 1, max [Abs(5S,—37;,)] (8.20) 
i=0 
where 


and 
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Spatial Filtering 


a. Original contrast stretched. b. Low-frequency filter. 


e. Maximum filter. 


i. Laplacian 4. j. Laplacian 5. 


k. Sobel edge enhancement. 


c. Median filter. 


1. Roberts edge enhancement. 


FIGURE 8-24 Application of various convolution masks and logic to the predawn thermal-infrared data to enhance low- and 
high-frequency detail: a) contrast stretched original image, b) low-frequency filter, c) median filter, d) minimum filter, e) maxi- 
mum filter, f) high-frequency filter, g) emboss NW filter, h) compass NE filter, i) Laplacian 4 filter, j) Laplacian 5 filter, k) Sobel 


edge enhancement, and |) Robert's edge enhancement. 


T; = BV 43+ BV ing + BVin5 + BV ing + BV 47. (8.22) 


The subscripts of BV are evaluated modulo 8, meaning 
that the computation moves around the perimeter of 
the mask in eight steps. The edge enhancement com- 
putes the maximal compass gradient magnitude about 
input image point BV; ;. The value of S; equals the sum 
of three adjacent pixels, while T; equals the sum of the 
remaining four adjacent pixels. The input pixel value at 
BV; ; is never used in the computation. 


For example, many of the linear and nonlinear spatial 
filtering convolution masks previously discussed were 
also applied to the thermal infrared image of the Sa- 
vannah River and displayed in Figure 8-24. Image ana- 
lysts usually work interactively with the remotely 
sensed data, trying different coefficients and selecting 
those that produce the most effective results. It is also 
possible to combine operators for edge detection. For 
example, a combination of gradient and Laplacian 
edge operators may yield results that are superior to 
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using either edge enhancement alone. In addition, non- 
linear edge enhancements may provide superior results. 


The Fourier Transform 


Fourier analysis is a mathematical technique for sepa- 
rating an image into its various spatial frequency com- 
ponents. First, let us consider a continuous function 
f(x). The Fourier theorem states that any function f(x) 
can be represented by a summation of a series of sinu- 
soidal terms of varying spatial frequencies. These 
terms can be obtained by the Fourier transform of 
f(x), which is written as: 


[peje 2a , 


F(u) = (8.23) 


where wu is spatial frequency. This means that F(w) is a 
frequency domain function. The spatial domain func- 
tion f(x) can be recovered from F(u) by the inverse 
Fourier transform: 


lo @) 
} F(uje2™"* dy , 


—0oo 


I(x) = (8.24) 


To use Fourier analysis in image processing, we must 
consider two extensions of these equations. First, both 
transforms can be extended from one-dimensional 
functions to two-dimensional functions f(x, y) and 
F(u, v). For Equation 8.23 this becomes: 


co 


[ | Panett Waedy. 


—oo 


F(u,v) = (8.25) 


Furthermore, we can extend both transforms to dis- 
crete functions. The two-dimensional discrete Fourier 
transform is written: 


N-1M=1—_api( ts 22) 
Fur) = YY faye 


ee 0 


(8.26) 


where JN is the number of pixels in the x direction and 
M is the number of pixels in the y direction. Every im- 
age can be described as a two-dimensional discrete 
function. Therefore, Equation 8.26 may be used to 
compute the Fourier transform of an image. The image 
can be reconstructed using the inverse transform: 


1 eee 
fey) = ‘ > Fu, ve . (8.27) 
u=O0v=0 


FIGURE 8-25 Digitized aerial photograph of The Loop on 
the Colorado River with three subimages identified. 


You are probably asking the question, “What does the 
F(u,v) represent?” It contains the spatial frequency in- 
formation of the original image f(x, y) and is called the 
frequency spectrum. Note that it is a complex function 
because it contains i, which equals /—1. We can write 
any complex function as the sum of a real part and an 
imaginary part: 


F(u, v) = R(u, v) + iu, v), (8.28) 
which is equivalent to 
F(u, v) = |F(u, vei), (8.29) 


where |F(u, v)| is a real function, and 


I R(u, v)2 + I(u, v)2. 


IF(u, v)| = 


|F(u, v)| is called the magnitude of the Fourier trans- 
form and can be displayed as a two-dimensional image. 
It represents the magnitude and the direction of the 
different frequency components in the image f(x, y). 
The variable @ in Equation 8.29 represents phase infor- 
mation in the image f(x, y). Although we usually ig- 
nore the phase information when we display the 


a. Low-frequency, homogeneous water 
body with no linear features. 


Fast 
Fourier 
Transform 
— 
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c. Low- and medium-frequency terrain with 
some horizontal and vertical linear features. 


e. Low- and medium-frequency terrain with 
some diagonal NW/SE linear features. 


FIGURE 8-26 Application of a Fourier transform to the three subimages of The Loop identified in Figure 8-25. 


Fourier transform, we cannot recover the original im- 
age without it. 


To understand how the Fourier transform is useful in 
remote sensing applications, let us first consider three 
subimages extracted from The Loop, shown in Figure 
8-25. The first subset includes a low-frequency water 
portion of the photograph shown enlarged in Figure 8- 


26a. Another area contains low- and medium-frequen- 
cy terrain information with both horizontal and verti- 
cal linear features (Figure 8-26c). The final subset 
contains low- and medium-frequency terrain with 
some diagonal linear features (Figure 8-26e). The mag- 
nitudes of the subimages’ Fourier transforms are 
shown in Figures 8-26b, d, and f. The Fourier magni- 
tude images are symmetric about their centers, and u 
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Stationary Periodic Noise and Its Fourier Transform 


Fast 
Fourier 
Transform 


— 


= 


a. High-frequency noise. 


c. Low-frequency noise. 


Fast 
Fourier 
Transform 


—<—>> 


b. Fourier transform. 


d. Fourier transform. 


FIGURE 8-27 Two examples of stationary periodic noise and their Fourier transforms. 


and vy represent spatial frequency. The displayed Fouri- 
er magnitude image is usually adjusted to bring the 
F(0, 0) to the center of the image rather than to the up- 
per-left corner. Therefore, the intensity at the center 
represents the magnitude of the lowest-frequency com- 
ponent. The frequency increases away from the center. 
For example, consider the Fourier magnitude of the 
homogeneous water body (Figure 8-26b). The very 
bright values found in and around the center indicate 
that it is dominated by low-frequency components. In 
the second image, more medium-frequency compo- 
nents are present in addition to the background of low- 
frequency components. We can relatively easily identify 
the high-frequency information representing the hori- 
zontal and vertical linear features in the original image 
(Figure 8-26d). Notice the alignment of the cloud of 
points in the center of the Fourier transform in Figure 
8-26f. It represents the diagonal linear features trend- 
ing in the NW-SE direction in the photograph. 


It is important to remember that the strange-looking 
Fourier transformed image F(u, v) contains all the in- 
formation found in the original image. It provides a 
mechanism for analyzing and manipulating images ac- 
cording to their spatial frequencies. It is useful for im- 
age restoration, filtering, and radiometric correction. 
For example, the Fourier transform can be used to re- 
move periodic noise in remotely sensed data. When the 
pattern of periodic noise is unchanged throughout the 
image, it is called stationary periodic noise. Striping in 
remotely sensed imagery is usually composed of sta- 
tionary periodic noise. 


When stationary periodic noise is a single-frequency si- 
nusoidal function in the spatial domain, its Fourier 
transform consists of a single bright point (a peak of 
brightness). For example, Figure 8-27a and c displays 
two images of sinusoidal functions with different fre- 
quencies (which look very much like striping in remote 
sensor data!). Figure 8-27b and d are their Fourier 


Landsat Thematic Mapper Data of Al Jubail, Saudi Arabia 
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September 1, 1990 
Band 4 


FIGURE 8-28 Landsat TM band 4 data of Al Jubail, Saudi Arabia, obtained on September 1, 1990. Image courtesy of NASA. 


transforms. The frequency and orientation of the noise 
can be identified by the position of the bright points. 
The distance from the bright points to the center of the 
transform (the lowest-frequency component in the im- 
age) is directly proportional to the frequency. A line 
connecting the bright point and the center of the trans- 
formed image is always perpendicular to the orienta- 
tion of the noise lines in the original image. Striping in 
the remotely sensed data is usually composed of sinu- 
soidal functions with more than one frequency in the 
same orientation. Therefore, the Fourier transform of 
such noise consists of a series of bright points lined up 
in the same orientation. 


Because the noise information is concentrated in a 
point or a series of points in the frequency domain, it is 
relatively straightforward to identify and remove them 
in the frequency domain, whereas it is quite difficult to 
remove them in the standard spatial domain. Basically, 
an analyst can manually cut out these lines or points in 
the Fourier transform image or use a computer pro- 
gram to look for such noise and remove it. For exam- 
ple, consider the Landsat TM band 4 data of Al Jubail, 
Saudi Arabia, obtained on September 1, 1990 (Figure 
8-28). The image contains serious stationary periodic 
striping, which can make the data unusable when con- 
ducting near-shore studies of suspended sediment 


transport. Figure 8-29 documents how a portion of the 
Landsat TM scene was corrected. First, a Fourier 
transform of the area was computed (Figure 8-29b). 
The analyst then modified the Fourier transform by se- 
lectively removing the points in the plot associated 
with the systematic striping (Figure 8-29c). This can be 
done manually or a special program can be written to 
look for and remove such systematic noise patterns in 
the Fourier transform image. The inverse Fourier 
transform was then computed, yielding a clean band 4 
image, which may be more useful for biophysical anal- 
ysis (Figure 8-29d). This type of noise could not be re- 
moved using a simple convolution mask. Rather, it 
requires access to the Fourier transform and selective 
editing out of the noise in the Fourier transform image. 


Figure 8-30 depicts hyperspectral data of the Mixed 
Waste Management Facility (MWMEF) on the Savan- 
nah River Site that was destriped using the same logic. 
A Fourier transform of a single band of data revealed 
the high-frequency information associated with the 
horizontal striping (Figure 8-30b). A cut filter was cre- 
ated to isolate the high-frequency information associ- 
ated with the horizontal striping (Figure 8-30c and d). 
The result of applying the cut filter and inverting the 
fast Fourier transform is shown in Figure 8-30e. The 
striping was reduced substantially. 
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Application of a Fourier Transform to Landsat Thematic Mapper Data 
of an Area Near Al Jubail, Saudi Arabia, to Remove Striping 


a. Original Landsat Thematic Mapper 
band 4 data obtained September 1, 1990. 


c. Interactively cleaned Fourier transform. 


b. Fourier transform. 


d. Corrected Landsat Thematic Mapper band 4 data. 


FIGURE 8-29 Application of a Fourier transform to a portion of Landsat TM band 4 data of Al Jubail, Saudi Arabia: a) original 
TM band 4 data, b) Fourier transform, c) cleaned Fourier transform, and d) destriped band 4 data. 


Spatial Filtering in Frequency Domain 

We have discussed filtering in the spatial domain using 
convolution filters. It can also be performed in the fre- 
quency domain. Using the Fourier transform, we can 
manipulate directly the frequency information of the 
image. The manipulation can be performed by multi- 
plying the Fourier transform of the original image by a 
mask image called a frequency domain filter, which 
will block or weaken certain frequency components by 
making the values of certain parts of the frequency 
spectrum become smaller or even zero. Then we can 
compute the inverse Fourier transform of the manipu- 
lated frequency spectrum to obtain a filtered image in 
the spatial domain. Numerous algorithms are available 
for computing the Fast Fourier transform (FFT) and 
the inverse Fast Fourier transform (IFFT) (Russ, 


2011). Spatial filtering in the frequency domain gener- 
ally involves computing the FFT of the original image, 
multiplying the FFT of a convolution mask of the ana- 
lyst’s choice (e.g., a low-pass filter) with the FFT, and 
inverting the resultant image with the IFFT; that is, 


f (x, y) FFT > F(u, v) > F(u, v) G(u, v) 
—> F'(u, v) IFFT > f (x, y). 
The convolution theorem states that the convolution of 
two images is equivalent to the multiplication of their 


Fourier transformations. If 


GY) = Jay)" 27). (8.30) 


CHAPTER 8 IMAGEENHANCEMENT 307 


Application of a Fourier Transform to Hyperspectral Data of the Savannah River Site 


a. Band 5 DAIS 3715 hyperspectral 
image (centered on 566 nm) of the 
MWMF at the Savannah River Site. 
Striping is noticeable in the 
enlarged image. 


e. Results of applying the cut filter 
and inverting the Fast Fourier 
transform. The before and after 
enlargement reveals that striping 
was substantially reduced. 


b. Fourier transform of c. Interactive identification of d. Final cut filter used 
the band 5 data. high-frequency noise. to remove the noise. 


FIGURE 8-30 Application of a Fourier transform to a single band of hyperspectral data of the Mixed Waste Management Fa- 
cility at the Savannah River Site near Aiken, SC (based on Jensen et al., 2003). 


where * represents the operation of convolution, where F'’, F, and Gare Fourier transforms of f’, f and 
f(x, y) is the original image and g(x,y) is aconvolution _ g, respectively. 
mask filter, then 
Two examples of such manipulation are shown in Fig- 
F'(u,v) = F(u, v)G(u, v), (8.31) ures 8-3la and b. A low-pass filter (Mask B) and a 
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Spatial Low-pass Filtering 
in the Frequency Domain 
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Low-pass filter 
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FIGURE 8-31 a) Spatial low-pass and b) high-pass filtering in the frequency domain using a Fourier transform. 


high-pass filter (Mask D) were used to construct the 
filter function g(x, y) in Figures 8-3la and b, respec- 
tively. In practice, one problem must be solved. Usual- 
ly, the dimensions of f(x, y) and g(x, y) are different; 
for example, the low-pass filter in Figure 8-3la has 
only nine elements, while the image is composed of 128 
x 128 pixels. Operation in the frequency domain re- 
quires that the sizes of F(u, v) and G(u, v) be the same. 
This means the sizes of fand g must be made the same 
because the Fourier transform of an image has the 
same size as the original image. The solution of this 
problem is to construct g(x, y) by putting the convolu- 
tion mask at the center of a zero-value image that has 
the same size as f: Note that in the Fourier transforms 
of the two convolution masks the low-pass convolution 
mask has a bright center (Figure 8-3la), while the 
high-pass filter has a dark center (Figure 8-31b). The 
multiplication of Fourier transforms F(u, v) and G(u, v) 
results in a new Fourier transform, F' ‘(u, v). Comput- 
ing the inverse Fast Fourier transformation produces 
i '(% y), a filtered version of the original image. Thus, 
spatial filtering can be performed in both the spatial 
and frequency domain.' 


As demonstrated, filtering in the frequency domain in- 
volves one multiplication and two transformations. For 
general applications, convolution in the spatial domain 
may be more cost-effective. Only when g(x, y) is very 
large does the Fourier method become cost-effective. 
However, with the frequency domain method we can 
also do some filtering that is not easy to do in the spa- 
tial domain. We may construct a frequency domain fil- 
ter G(u,v) specifically designed to remove certain 
frequency components in the image. 


| Principal Components 
Analysis (PCA) 


Principal components analysis (often called PCA, or 
Karhunen—Loeve analysis) has proven to be of great 
value in the analysis of multispectral and hyperspectral 
remotely sensed data (e.g., Mitternicht and Zinck, 
2003; Amato et al., 2008; Small, 2012; Mberego et al., 
2013). Principal components analysis is a technique 
that transforms the original remotely sensed dataset 
into a substantially smaller and easier to interpret set 


TABLE 8-5 Charleston, SC, Thematic Mapper scene statistics used in the principal components analysis (PCA). 
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Band Number: 1 2 3 4 5 7 6 
Xm 0.45-0.52 0.52-0.60 0.63-0.69 0.76-0.90 1.55-1.75 2.08-2.35 10.4-12.5 
Univariate Statistics 
Mean 64.80 25.60 23.70 27.30 32.40 15.00 110.60 
Standard Deviation 10.05 5.84 8.30 15.76 23.85 12.45 4.21 
Variance 100.93 34.14 68.83 248.40 568.84 154.92 17.78 
Minimum 51 17 14 4 0 0 90 
Maximum 242 5 131 105 193 128 130 
Variance—Covariance Matrix 
1 100.93 
2 56.60 34.14 
3} 79.43 46.71 68.83 
4 61.49 40.68 69.59 248.40 
5 134.27 85.22 141.04 330.71 568.84 
7 FOALS 55.14 86.91 148.50 280.97 154.92 
6 ERIE 14.33 22.92 43.62 78.91 42.65 17.78 
Correlation Matrix 
1 1.00 
2 0.96 1.00 
3 0.95 0.96 1.00 
4 0.39 0.44 0.53 1.00 
5 0.56 0.61 0.71 0.88 1.00 
7 0.72 0.76 0.84 0.76 0.95 1.00 
6 0.56 0.58 0.66 0.66 0.78 0.81 1.00 


of uncorrelated variables that represents most of the 
information present in the original dataset (Good et 
al., 2012). Principal components are extracted from 
the original data such that the first principal compo- 
nent accounts for the maximum proportion of the 
variance of the original dataset, and subsequent or- 
thogonal components account for the maximum pro- 
portion of the remaining variance (Zhao and Maclean, 
2000; Viscarra-Rossel and Chen, 2011). The ability to 
reduce the dimensionality (i.e., the number of bands in 
the dataset that must be analyzed to produce usable 
results) from 7 to just a few bands is an important eco- 
nomic consideration, especially if the potential infor- 
mation that can be recovered from the transformed 
data is just as good as the original remote sensor data. 


A form of PCA may also be useful for reducing the di- 
mensionality of hyperspectral datasets. For example, 
many scientists use a minimum noise fraction (MNF) 
procedure, which is a cascaded principal components 
analysis for data compression and noise reduction of 
hyperspectral data (Jensen et al., 2003). Chapter 11 
provides an example of the use of the MNF procedure. 


To perform principal components analysis we apply a 
transformation to a correlated set of multispectral da- 
ta. For example, the Charleston TM scene is a likely 
candidate because bands 1, 2, and 3 are highly correlat- 
ed, as are bands 5 and 7 (Table 8-5). The application of 
the transformation to the correlated remote sensor 
data will result in an uncorrelated multispectral dataset 
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Principal Components Analysis 
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FIGURE 8-32 Diagrammatic representation of the spatial relationship between the first two principal components. a) Scatter- 
plot of data points collected from two remotely sensed bands labeled X, and X2 with the means of the distribution labeled & 
and &b. b) A new coordinate system is created by shifting the axes to an X’ system. The values for the new data points are 
found by the relationship X,° = X, — &, and Xx" = Xz — Mb. c) The X’ axis system is then rotated about its origin (), Ky) so that 
PC, is projected through the semi-major axis of the distribution of points and the variance of PC, is a maximum. PC> must be 
perpendicular to PC;. The PC axes are the principal components of this two-dimensional data space. Principal component 1 
often accounts for > 90% of the variance, and principal component 2 accounts for 2% to 10%, etc. 


that has certain ordered variance properties. This 
transformation is conceptualized by considering the 
two-dimensional distribution of pixel values obtained 
in two TM bands, which we will label X; and X>. A 
scatterplot of all the brightness values associated with 
each pixel in each band is shown in Figure 8-32a, along 
with the location of the respective means, WM, and XM). 
The spread or variance of the distribution of points is 
an indication of the correlation and quality of informa- 
tion associated with both bands. If all the data points 
were clustered in an extremely tight region in the two- 
dimensional space, these data would probably provide 
very little information. 


The initial measurement coordinate axes (X, and X>) 
may not be the best arrangement in multispectral fea- 
ture space to analyze the remote sensor data associated 
with these two bands. The goal is to use principal com- 
ponents analysis to translate and/or rotate the original 
axes so that the original brightness values on axes Xy 
and X, are redistributed (reprojected) onto a new set of 
axes or dimensions, X”’; and X’y. For example, the best 
translation for the original data points from X, to X7; 
and from X> to X’> coordinate systems might be the 
simple relationship X’,; = X; —&, and X’y = X> — wb. 
Thus, the new coordinate system origin (X’; and X’5) 
now lies at the location of both means in the original 
scatter of points (Figure 8-32b). The X’ coordinate sys- 
tem might then be rotated about its new origin (X), X) 
in the new coordinate system some ¢ degrees so that 
the first axis X’, is associated with the maximum 
amount of variance in the scatter of points (Estes et al., 


2010) (Figure 8-32c). This new axis is called the first 
principal component (PC, = i 1). The second principal 
component (PC, = i>) is perpendicular (orthogonal) 
to PC. Thus, the major and minor axes of the ellip- 
soid of points in bands X; and_X, are called the princi- 
pal components. The third, fourth, fifth, and so on, 
components contain decreasing amounts of the vari- 
ance found in the dataset. 


To transform (reproject) the original data on the X, 
and X> axes onto the PC; and PC), axes, we must ob- 
tain certain transformation coefficients that we can ap- 
ply in a linear fashion to the original pixel values. The 
linear transformation required is derived from the co- 
variance matrix of the original dataset. Thus, this is a 
data-dependent process with each new remote sensing 
dataset yielding different transformation coefficients. 


The transformation is computed from the original 
spectral statistics, as follows (Short, 1982): 


1. The » X n covariance matrix, Cov, of the n- 
dimensional remote sensing dataset to be transformed 
is computed (Table 8-5). Use of the covariance matrix 
results in a non-standardized PCA, whereas use of the 
correlation matrix results in a standardized PCA 
(Eastman and Fulk, 1993; Carr, 1998). 


The eigenvalues, E = [A 11, 429, 433, --» Anal, and 
eigenvectors, EV = [a;, ... for k = 1 ton bands, and p = 
1 to n components], of the covariance matrix are com- 
puted such that: 


TABLE 8-6 Eigenvalues computed for the covariance matrix. 
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Component p 
1 2 3 4 5 6 7 
Eigenvalues, 2» 1,010.92 | 131.20 37.60 6.73 3.95 247 1.24 
Difference 879.72 93.59 30.88 Zail, Ned/7/ 83 -- 
Total Variance = 1193.81 
Percent of total variance in the data explained by each component: 
eigenvalue 2 x 100 
Computed as %_ = a ee 
- 8 
> eigenvalue i, 
p=l 
For example, 
7 
> A» = 1,010.92 + 131.20 + 37.60 + 6.73 + 3.95 + 2.17 + 1.24 1,193.81. 
p=l 
Percentage of variance explained by first component = ee = 84.68. 
Percentage 84.68 10.99 Sal) 0.56 0.33 0.18 0.10 
Cumulative 84.68 95.67 98.82 99.38 PI 99.89 100.00 
E variance explained by each of the _ principal 
components, %,, using the equation: 
Ai, 09 0 0 0 0 of] 
eigenvalue 4 x 100 
0 ry, 0 0 80 0 0 a ee ay (8.33) 
0 0 433 0 0 0 0 . ib 
‘ eigenvalue » 
EV-Cov-EV. =| 0 0 0 hag 9 0 0 | (8.32) 2; P 
: eal 
0 0 0 0 Ass 0 0 
0 0 0 0 OA 0 j : : ; 
6,6 where i , is the pth eigenvalue out of the possible n ei- 
rr rr | genvalues. For example, the first principal component 


where EV! is the transpose of the eigenvector matrix, 
EV, and E is a diagonal covariance matrix whose ele- 
ments A ;;, called eigenvalues, are the variances of the 
pth principal components, where p = | to n components. 
The nondiagonal eigenvalues, d ip are equal to zero 
and therefore can be ignored. The number of nonzero 
eigenvalues in an n Xn covariance matrix always equals 
n, the number of bands examined. The eigenvalues are 
often called components (i.e., eigenvalue 1 may be re- 
ferred to as principal component 1). Eigenvalues and 
eigenvectors were computed for the Charleston, SC, 
Thematic Mapper scene (Tables 8-6 and 8-7). 


The eigenvalues contain important information. For 
example, it is possible to determine the percent of total 


(eigenvalue 2) of the Charleston TM scene accounts 
for 84.68% of the variance in the entire multispectral 
dataset (Table 8-6). Component 2 accounts for 10.99% 
of the remaining variance. Cumulatively, these first two 
principal components account for 95.67% of the vari- 
ance. The third component accounts for another 
3.15%, bringing the total to 98.82% of the variance ex- 
plained by the first three components (Table 8-6). 
Thus, the seven-band TM dataset of Charleston might 
be compressed into just three new principal component 
images (or bands) that explain 98.82% of the variance. 


But what do these new components represent? For ex- 
ample, what does component | stand for? By comput- 
ing the correlation of each band k with each 
component p, it is possible to determine how each 
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TABLE 8-7 Eigenvectors (Ag) (factor scores) computed for the covariance matrix found in Table 8-5. 


Component p 
1 2 s} 4 5 6 7 
band; 1 0.205 0.637 0.327 -0.054 0.249 -0.611 -0.079 
2 0.127 0.342 0.169 -0.077 0.012 0.396 0.821 
3 0.204 0.428 0.159 -0.076 -0.075 0.649 -0.562 
4 0.443 -0.471 0.739 0.107 -0.153 -0.019 0.004 
5 0.742 -0.177 0.437 0.300 0.370 0.007 0.011 
7 0.376 0.197 0.309 0.312 -0.769 -0.181 0.051 
6 0.106 0.033 0.080 0.887 0.424 0.122 0.005 
band “loads” or is associated with each principal com- _ will evaluate the first pixel in a hypothetical image at 
ponent. The equation is: row 1, column 1 for each of seven bands. We will repre- 
sent this as the vector X, such that 
a & dh 2 7 
k 
Ruy = “ip alhy (8.34) BV, , = 20 
Var oi, 
here ‘ BV = 30 
~ i= 
Ay = eigenvector for band & and component p B ‘ae 1,37 = 
i » = pth eigenvalue X= |B an iat 60 
Var, = variance of band k in the covariance matrix. BV, 1. 5= 70 
This computation results in a new n Mn matrix (Table B an ia 62 
8-8) filled with factor loadings. For example, the high- BV, 1 6= 50 


est correlations (i.c., factor loadings) for principal 
component | were for bands 4, 5, and 7 (0.894, 0.989, 
and 0.961, respectively; Table 8-8). This suggests that 
this component is a near- and middle-infrared reflec- 
tance band. This makes sense because the golf courses 
and other vegetation are particularly bright in this im- 
age (Figure 8-33). Conversely, principal component 2 
has high loadings only in the visible bands 1, 2, and 3 
(0.726, 0.670, and 0.592), and vegetation is noticeably 
darker in this image (Figure 8-33). This is a visible 
spectrum component. Component 3 loads heavily on 
the near-infrared (0.287) and appears to provide some 
unique vegetation information. Component 4 accounts 
for little of the variance but is easy to label since it 
loads heavily (0.545) on the thermal-infrared band 6. 
Components 5, 6, and 7 provide no useful information 
and contain most of the systematic noise. They ac- 
count for very little of the variance and should proba- 
bly not be used further. 


Now that we understand what information each com- 
ponent contributes, it is useful to see how the principal 
component images are created. To do this, it is neces- 
sary to first identify the original brightness values 
(BV; ;,) associated with a given pixel. In this case we 


We will now apply the appropriate transformation to 

this data such that they are projected onto the first 

principal component’s axes. In this way we will find out 

what the new brightness value (new BY; ; ,) will be for 

this component, p. It is computed using the equation: 
n 


new RY ag DAY ee 
k=1 


(8.35) 


where a; = eigenvectors, BV; ;; = brightness value in 
band k for the pixel at row i, column j, and m = number 
of bands. In our hypothetical example, this yields: 
Dey fet he i a eg a 
Baa ia GN tal es he ee 
6 1 ONG 1 7 1 
= 0.205(20) + 0.127(30) + 0.204(22) + 0.443(60) + 


0.742(70) + 0.376(62) + 0.106(50) 


= 119.53 
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TABLE 8-8 Degree of correlation, R,,,, between each band k and each principal component p. 


Computed: 


For example: 
0.205 x J1,010.92 _ 0.205 x 31.795 


Sh e (100.93 Wet ok 
R.. = 742 x /101092 _ 0.742 x 31.795 _ 9 o99 
5,1 (568.84 23.85 
Fe WEEER SEIT UL 
2,2 (34.14 5.842 
Component p 
1 2 3 4 5 6 z, 
Band, 1 0.649 0.726 0.199 -0.014 0.049 -0.089 -0.008 
2 0.694 0.670 0.178 -0.034 0.004 0.099 0.157 
3 0.785 0.592 0.118 -0.023 -0.018 0.115 -0.075 
4 0.894 -0.342 0.287 0.017 -0.019 -0.002 -0.000 
5 0.989 -0.084 =o 112 -0.032 0.030 0.000 0.000 
7 0.961 0.181 0.152 0.065 -0.122 -0.021 0.004 
6 0.799 0.089 0.116 0.545 0.200 0.042 0.001 


This pseudo-measurement is a linear combination of 
original brightness values and factor scores (eigenvec- 
tors). The new brightness value for row 1, column | in 
principal component | after truncation to an integer is 
new BViiy = 119. 


This procedure takes place for every pixel in the origi- 
nal remote sensor image data to produce the principal 
component | image dataset. Then p is incremented by 
1 and principal component 2 is created pixel by pixel. 
This is the method used to produce the principal com- 
ponent images shown in Figure 8-33a—g. If desired, any 
two or three of the principal components can be placed 
in the blue, green, and/or red graphics display channels 
to create a principal component color composite. For 
example, principal components 1, 2, and 3 are dis- 
played as an RGB color composite in Figure 8-331. 
These displays often depict more subtle differences in 
color shading and distribution than in the traditional 
color-infrared color composite image. 


If components 1, 2, and 3 account for a significant 
amount of the variance in the dataset, perhaps the 
original seven bands of TM data can be set aside, and 
the remainder of the image enhancement or classifica- 
tion can be performed using just these three principal 
component images. This reduces the amount of data to 


be analyzed and completely bypasses the expensive and 
time-consuming process of feature (band) selection so 
often necessary when classifying remotely sensed data 
(discussed in Chapter 9). 


Eastman and Fulk (1993) suggested that standardized 
PCA (based on the computation of eigenvalues from 
correlation matrices) is superior to non-standardized 
PCA (computed from covariance matrices) when ana- 
lyzing change in multitemporal image datasets. They 
processed 36 monthly AVHRR-derived normalized 
difference vegetation index (NDVI) images of Africa 
for the years 1986 to 1988. They found the first compo- 
nent was always highly correlated with NDVI regard- 
less of season, while the second, third, and fourth 
components related to seasonal changes in NDVI. 
Mitternicht and Zinck (2003) found PCA very useful 
for delineating saline from nonsaline soils. Amato et al. 
(2008) used PCA and independent component analysis 
(ICA) to perform cloud detection using European Me- 
teosat data. Good et al. (2012) used PCA in their de- 
velopment of the ATSR Saharan Dust Index (ASDI) 
based on the use of thermal infrared imagery from the 
Along-Track Scanning Radiometers (ATSRs). Inter- 
estingly, the first component provided clear-sky infor- 
mation while the second component was found to be 
the ASDI. Estes et al. (2010) developed a PCA-derived 
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Principal Component Images of Charleston, SC, Derived from 


Landsat Thematic Mapp 


a p ae 


g. Principal component 7. 


er Imagery Obtained on November 9, 1982 


— ai _* 


= 


h. RGB = Landsat TM bands 4,3,2. 


FIGURE 8-33 a-g) Seven principal component images of the Charleston Thematic Mapper data computed using all seven 
bands. Component 1 consists of both near- and middle-infrared information (bands 4, 5, and 7). Component 2 contains pri- 
marily visible light information (bands 1, 2, and 3). Component 3 contains primarily near-infrared information. Component 4 
consists of the thermal infrared information contributed by band 6. Thus, the seven-band TM data can be reduced in dimen- 
sion to just four principal components (1, 2, 3, and 4), which account for 99.38% of the variance. h) A color-infrared color com- 
posite consisting of the original Landsat TM bands 4, 3, and 2 is displayed. i) A color composite of principal component bands 


1,2, and 3. Original imagery courtesy of NASA. 


Canopy Structure Index (CSI) for modeling habitat 
structural complexity. Viscarra-Rosel and Chen (2011) 
used PCA to analyze visible and near-infrared spectra 
and other environmental variables to map soils in Aus- 
tralia. Small (2012) used PCA and temporal mixture 
modeling to perform time-space characterization of 
multiple dates of MODIS Enhanced Vegetation Index 
(EVI) data (discussed in the next section) of the Gan- 
ges Brahmaputra delta in India. 


(ey Vegetation Indices (VI) 


Approximately 70% of the Earth’s land surface is cov- 
ered with vegetation. Knowledge about variations in 
vegetation species and community distribution pat- 
terns, alterations in vegetation phenological (growth) 
cycles, and modifications in the plant physiology and 
morphology provides valuable insight into the climatic, 
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Spectral Characteristics of Healthy Green Vegetation 
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FIG RE 8-34 Spectral reflectance characteristics of healthy, green vegetation for the wavelength interval 0.4 - 2.6 Xm. The 
dominant factors controlling leaf reflectance are the various leaf pigments in the palisade mesophyll (e.g., chlorophyll a and 
b, and  -carotene), the scattering of near-infrared energy in the spongy mesophyll, and the amount of water in the plant. 
The primary chlorophyll absorption bands occur at 0.43 - 0.45 Xm and 0.65 — 0.66 Km in the visible region. The primary water 
atmospheric absorption bands occur at 0.97, 1.19, 1.45, 1.94, and 2.7 Km. 


edaphic, geologic, and physiographic characteristics of 
an area (e.g., Jackson and Jensen, 2005; Im and Jensen, 
2009). Furthermore, the goal of global agriculture pro- 
duction and the grain sector of most economies is to 
feed billions of people. Therefore, it is not surprising 
that scientists have devoted significant effort to develop 
remote sensing systems and visual and digital image 
processing algorithms to extract important vegetation 
biophysical information from remotely sensed data. 
Many of the remote sensing techniques are generic in 
nature and may be applied to a variety of vegetated 
landscapes, including: 


* agriculture, 

¢ forests, 

* rangeland, 

¢ wetland, and 

* manicured urban vegetation. 


This section introduces the fundamental concepts as- 
sociated with vegetation biophysical characteristics 
and how remotely sensed data can be processed to pro- 
vide unique information about these parameters. It 
then summarizes some of the vegetation indices devel- 
oped to extract biophysical vegetation information 
from digital remote sensor data. 


Before presenting the various vegetation indices, how- 
ever, it is important to understand how the indices are 
related to various leaf physiological properties. There- 
fore, we will first discuss the dominant factors control- 
ling leaf reflectance (Figure 8-34). This will help us 
appreciate why the linear combination of certain bands 
can be used in vegetation indices as a surrogate for leaf 
and/or canopy biophysical properties. 
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Dominant Factors Controlling Leaf 
Reflectance 


Pioneering work by Gates et al. (1965), Gausmann et 
al. (1969), Myers (1970) and others demonstrated the 
importance of understanding how leaf pigments, inter- 
nal scattering, and leaf water content affect the reflec- 
tance and transmittance properties of leaves (Peterson 
and Running, 1989). Dominant factors controlling leaf 
reflectance in the region from 0.35 — 2.6 Km are sum- 
marized in Figure 8-34. 


Visible Light Interaction with Pigments in the 
Palisade Mesophyll Cells 

Photosynthesis is an energy-storing process that takes 
place in leaves and other green parts of plants in the 
presence of light. The photosynthetic process is: 


6CO, + 6H,O + light energy > C5H),0, + 60,. 


Sunlight provides the energy that powers photosynthe- 
sis. The light energy is stored in a simple sugar mole- 
cule (glucose) that is produced from carbon dioxide 
(CO >) present in the air and water (H,O) absorbed by 
the plant primarily through the root system. When the 
carbon dioxide and the water are combined and form a 
sugar molecule (CgH,O¢) in a chloroplast, oxygen gas 
(O>) is released as a by-product. The oxygen diffuses 
out into the atmosphere. 


The photosynthetic process begins when sunlight 
strikes chloroplasts, which are small bodies in the leaf 
that contain a green substance called chlorophyll. It is 
the process of food-making via photosynthesis that de- 
termines how a leaf and the associated plant canopy 
components appear on remotely sensed images. 


Plants have adapted their internal and external struc- 
ture to perform photosynthesis. This structure and its 
interaction with electromagnetic energy have a direct 
impact on how leaves and canopies appear spectrally 
when recorded using remote sensing instruments. 


The leaf is the primary photosynthesizing organ. A 
cross-section of a typical green leaf is shown in Figure 
8-35. The cell structure of leaves is highly variable de- 
pending upon species and environmental conditions 
during growth. Carbon dioxide enters the leaf from the 
atmosphere through tiny pores called stomata or sto- 
ma, located primarily on the underside of the leaf on 
the lower epidermis. Each stomata is surrounded by 
guard cells that swell or contract. When they swell, the 
stomata pore opens and allows carbon dioxide to enter 
the leaf. A typical sunflower leaf might have 2 million 
stomata, but they make up only about | percent of the 
leaf’s surface area. Usually, there are more stomata on 
the bottom of a leaf; however, on some leaves the sto- 


mata are evenly distributed on both the upper and low- 
er epidermis. 


The top layer of leaf upper epidermis cells has a cuticu- 
lar surface that diffuses but reflects very little light. It is 
variable in thickness but is often only 3 — 5 Mm thick 
with cell dimensions of approximately 18 x15 x 20 Xm. 
It is useful to think of it as a waxy, translucent material 
similar to the cuticle at the top of your fingernail. 
Leaves of many plants that grow in bright sunlight 
have a thick cuticle that can filter out some light and 
guard against excessive plant water loss. Conversely, 
some plants such as ferns and some shrubs on the for- 
est floor must survive in shaded conditions. The leaves 
of many of these plants have a thin cuticle so that the 
plant can collect as much of the dim sunlight as possi- 
ble for photosynthesis. 


Many leaves in direct sunlight have hairs growing out 
of the upper (and lower) epidermis, causing them to 
feel fuzzy. These hairs can be beneficial, as they reduce 
the intensity of the incident sunlight to the plant. Nev- 
ertheless, much of the visible and near-infrared wave- 
length energy is transmitted through the cuticle and 
upper epidermis to the palisade parenchyma mesophyll 
cells and spongy parenchyma mesophyll cells below. 


Photosynthesis occurs inside the typical green leaf in 
two kinds of food-making cells—palisade parenchyma 
and spongy parenchyma mesophyll cells. Most leaves 
have a distinct layer of long palisade parenchyma cells 
in the upper part of the mesophyll and more irregularly 
shaped, loosely arranged spongy parenchyma cells in 
the lower part of the mesophyll. The palisade cells tend 
to form in the portion of the mesophyll toward the side 
from which the light enters the leaf. In most horizontal 
(planophile) leaves the palisade cells will be toward the 
upper surface, but in leaves that grow nearly vertical 
(erectophile), the palisade cells may form from both 
sides. In some leaves the elongated palisade cells are 
entirely absent and only spongy parenchyma cells will 
exist within the mesophyll. 


The cellular structure of the leaf is large compared to 
the wavelengths of light that interact with it. Palisade 
cells are typically 15 x 15 x 60 Xm, while spongy meso- 
phyll parenchyma cells are smaller. The palisade paren- 
chyma mesophyll plant cells contain chloroplasts with 
chlorophyll pigments. 


The chloroplasts are generally 5 — 8 Xm in diameter 
and about | Xm in width. As many as 50 chloroplasts 
may be present in each parenchyma cell. Within the 
chloroplasts there are long, slender grana strands (not 
shown in Figure 8-35) within which the chlorophyll is 
actually located (approximately 0.5 Xm in length and 
0.05 Xm in diameter). The chloroplasts are generally 
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FIGURE 8-35 a) Hypothetical cross-section of a healthy green leaf showing both the top and underside of the leaf. The chlo- 
rophyll pigments in the palisade parenchyma cells have a significant impact on the absorption and reflectance of visible light 
(blue, green, and red), while the spongy parenchyma mesophyll cells have a significant impact on the absorption and reflec- 
tance of near-infrared incident energy. b) Electron microscope image of a green leaf. 


more abundant toward the upper side of the leaf in the 
palisade cells and hence account for the darker green 
appearance of the upper leaf surface compared with 
the bottom lighter surface. 


A molecule, when struck by a photon of light, reflects 
some of the energy or it can absorb the energy and thus 
enter into a higher energy or excited state (refer to 


Chapter 6). Each molecule absorbs or reflects its own 
characteristic wavelengths of light. Molecules in a typi- 
cal green plant have evolved to absorb wavelengths of 
light in the visible region of the spectrum (0.35 — 0.70 
im) very well and are called pigments. An absorption 
spectrum for a particular pigment describes the wave- 
lengths at which it can absorb light and enter into an 
excited state. Figure 8-36a presents the absorption 
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FIGURE 8-36 a) Absorption spectra of chlorophyll a and b pigments. Chlorophyll a and b pigments in the leaf absorb much 
of the incident blue and red wavelength energy: chlorophyll a at wavelengths centered on 0.43 and 0.66 Mm and chlorophyll 
b at wavelengths centered on 0.45 and 0.65 Xm. b) Absorption spectra of B -carotene, which absorbs primarily in the blue 
has a strong absorption band centered at about 0.45 Xm. Other pigments that might be found in a leaf include Phycoerythrin 
which absorbs primarily green light, and Phycocyanin which absorbs primarily green and red light. 


spectrum of pure chlorophyll pigments in solution. 
Chlorophyll a and b are the most important plant pig- 
ments absorbing blue and red light: chlorophyll a at 
wavelengths of 0.43 and 0.66 Xm and chlorophyll } at 
wavelengths of 0.45 and 0.65 Xm. A relative lack of ab- 
sorption in the wavelengths between the two chloro- 
phyll absorption bands produces a trough in the 
absorption efficiency at approximately 0.54 Kim in the 
green portion of the electromagnetic spectrum (Figure 
8-36a). Thus, it is the relatively lower absorption of 
green wavelength light (compared to blue and red 
light) by the leaf that causes healthy green foliage to 
appear green to our eyes. 


There are other pigments present in the palisade meso- 
phyll cells that are usually masked by the abundance of 
chlorophyll pigments. For example, there are yellow 
carotenes and pale yellow xanthophyll pigments, with 
strong absorption primarily in the blue wavelength re- 
gion. The B-carotene absorption spectra is shown in 
Figure 8-36b with its strong absorption band centered 
at about 0.45 Xm. Phycoerythrin pigments may also be 
present in the leaf, which absorb predominantly in the 
green region centered at about 0.55 Xm, allowing blue 
and red light to be reflected. Phycocyanin pigments ab- 
sorb primarily in the green and red regions centered at 
about 0.62 Km, allowing much of the blue and some of 
the green light (i.e., the combination produces cyan) to 
be reflected (Figure 8-36b). Because chlorophyll a and 
b chloroplasts are also present and have a similar ab- 
sorption band in this blue region, they tend to domi- 
nate and mask the effect of the other pigments present. 


When a plant undergoes senescence in the fall or en- 
counters stress, the chlorophyll pigment may disap- 
pear, allowing the carotenes and other pigments to 
become dominant. For example, in the fall, chlorophyll 
production ceases, causing the yellow coloration of the 
carotenes and other specific pigments in the tree foliage 
to become more visible to our eyes. In addition, some 
trees produce great quantities of anthocyanin in the 
fall, causing the leaves to appear bright red. 


The two optimum spectral regions for sensing the chlo- 
rophyll absorption characteristics of a leaf are believed 
to be 0.45 — 0.52 Xm and 0.63 — 0.69 Xm (Figure 8-36a). 
The former region is characterized by strong absorp- 
tion by carotenoids and chlorophylls, whereas the lat- 
ter is characterized by strong chlorophyll absorption. 
Remote sensing of chlorophyll absorption within a 
canopy represents a fundamental biophysical variable 
useful for many biogeographical investigations. The 
absorption characteristics of plant canopies may be 
coupled with other remotely sensed data to identify 
vegetation stress, yield, and other hybrid variables. 
Thus, many remote sensing studies are concerned with 
monitoring what happens to the photosynthetically ac- 
tive radiation (PAR) as it interacts with individual 
leaves and/or the plant canopy. High spectral resolu- 
tion imaging spectrometers are useful for measuring 
the absorption and reflectance characteristics of the 
photosynthetically active radiation. To demonstrate 
these principles, consider the spectral reflectance char- 
acteristics of four different leaves obtained from a sin- 
gle healthy Sweetgum tree (Liguidambar styraciflua L.) 
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FIGU RE 8-37 a) Photosynthesizing green Sweetgum leaf (Liquidambar sty raciflua L.) obtained from a tree on November 11, 
1998. b-c) Senescing yellow and red Sweetgum leaves obtained on the same tree. d) Senesced Sweetgum leaf that was on 
the ground. e) Spectroradiometer percent reflectance measurements over the wavelength interval 400 - 1,050 nm. 


on November 11, 1998, in Columbia, SC (Figure 8-37). 
The green leaf (a), yellow leaf (b), and red leaf (c) were 
collected directly from the Sweetgum tree. The dark 
brown leaf (d) was collected on the ground beneath the 
tree. 


A GER 1500 (Geophysical & Environmental Research, 
Inc.) handheld spectroradiometer was used to obtain 
spectral reflectance measurements from each of the 
leaves. The spectroradiometer obtained spectral reflec- 
tance measurements in 512 bands in the ultraviolet, 
blue, green, red, and near-infrared spectral regions 


from 350—1,050 nm. Percent reflectance measurements 
were obtained in the lab by measuring the amount of 
energy reflected from the surface of the leaf (the target) 
divided by the amount of energy reflected from a Spec- 
tralonfi reflectance reference (percent reflectance = 
target/reference X 100). The reflectance measurements 
for each of the leaves from 400—1,050 nm were plotted 
in a percent reflectance graph (Figure 8-37). 


The green leaf (Figure 8-37a) was still photosynthesiz- 
ing and yielded a typical healthy green reflectance 
spectra with strong chlorophyll absorption bands in 
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the blue and red regions (approximately 6% reflectance 
at 450 nm and 5% at 650 nm, respectively), and a peak 
in reflectance in the green region of the visible spec- 
trum (11% at 550 nm). Approximately 76% of the inci- 
dent near-infrared radiant flux was reflected from the 
leaf at 900 nm. 


The yellow leaf (Figure 8-37b) was undergoing senes- 
cence. As the influence of the chlorophyll pigments di- 
minished, relatively greater amounts of green (24% at 
550 nm) and red (32% at 650 nm) light were reflected 
from the leaf, resulting in a yellow appearance. At 750 
nm, the yellow leaf reflected less near-infrared radiant 
flux than the healthy green leaf. Conversely, the near- 
infrared reflectance at 900 nm was about 76%, very 
similar to the healthy green leaf. 


The red leaf (Figure 8-37c) reflected 7% of the blue at 
450 nm, 6% of the green energy at 550 nm, and 23% of 
the incident red energy at 650 nm. Near-infrared reflec- 
tance at 900 nm dropped to 70%. 


The dark brown leaf (Figure 8-37d) produced a spec- 
tral reflectance curve with low blue (7% at 450 nm), 
green (9% at 550 nm), and red reflectance (10% at 650 
nm). This combination produced a dark brown ap- 
pearance. Near-infrared reflectance dropped to 44% at 
900 nm. 


It is important to understand the physiology of the 
plants under investigation and especially their pigmen- 
tation characteristics so that we can appreciate how a 
typical plant will appear when chlorophyll absorption 
starts to decrease, either due to seasonal senescence or 
environmental stress. As demonstrated, when a plant is 
under stress and/or chlorophyll production decreases, 
the lack of chlorophyll pigmentation typically causes 
the plant to absorb less in the chlorophyll absorption 
bands. Such plants will have a much higher reflectance, 
particularly in the green and red portion of the spec- 
trum, and therefore may appear yellowish or chlorotic. 
In fact, Carter (1993) suggests that increased reflec- 
tance in the visible spectrum is the most consistent leaf 
reflectance response to plant stress. Infrared reflec- 
tance responds consistently only when stress has devel- 
oped sufficiently to cause severe leaf dehydration (to be 
discussed shortly). 


Leaf spectral reflectance is most likely to indicate plant 
stress first in the sensitive 535 — 640 and 685 — 700 nm 
visible light wavelength ranges. Increased reflectance 
near 700 nm represents the often reported “blue shift 
of the red edge,” 1.e., the shift toward shorter wave- 
lengths of the red-infrared transition curve that occurs 
in stressed plants when reflectance is plotted versus 
wavelength (Cibula and Carter, 1992). The shift toward 
shorter wavelengths in the region from 650 — 700 nm is 


particularly evident for the yellow and red reflectance 
curves shown in Figure 8-37e. Remote sensing within 
these spectrally narrow ranges may provide improved 
capability to detect plant stress not only in individual 
leaves but for whole plants and perhaps for densely 
vegetated canopies (Carter, 1993; Carter et al., 1996). 


Normal color film is sensitive to blue, green, and red 
wavelength energy. Color-infrared film is sensitive to 
green, red, and near-infrared energy after minus-blue 
(yellow) filtration. Therefore, even the most simple 
camera with color or color-infrared film and appropri- 
ate band-pass filtration (i.e, only certain wavelengths 
of light are allowed to pass) can be used to remotely 
sense differences in spectral reflectance caused by the 
pigments present in the palisade parenchyma layer of 
cells in a typical leaf. However, to detect very subtle 
spectral reflectance differences in the relatively narrow 
bands suggested by Cibula and Carter (1992) and 
Carter et al. (1996), it may be necessary to use a high 
spectral resolution imaging spectroradiometer that has 
very narrow bandwidths. 


Near-Infrared Energy Interaction within the 
Spongy Mesophyll Cells 

In a typical healthy green leaf, the near-infrared reflec- 
tance increases dramatically in the region from 700 — 
1,200 nm. For example, the healthy green leaf in the 
previous example reflected approximately 76% of the 
incident near-infrared energy at 900 nm. Healthy green 
leaves absorb radiant energy very efficiently in the blue 
and red portions of the spectrum where incident light 
is required for photosynthesis. But immediately to the 
long wavelength side of the red chlorophyll absorption 
band, why do the reflectance and transmittance of 
plant leaves increase so dramatically, causing the ab- 
sorptance to fall to low values (Figure 8-34)? This con- 
dition occurs throughout the near-infrared wavelength 
range where the direct sunlight incident on plants has 
the bulk of its energy. If plants absorbed this energy 
with the same efficiency as they do in the visible region, 
they could become much too warm and the proteins 
would be irreversibly denatured. As a result, plants 
have adapted so they do not use this massive amount 
of near-infrared energy and simply reflect it or trans- 
mit it through to underlying leaves or the ground. 


The spongy mesophyll layer in a green leaf controls the 
amount of near-infrared energy that is reflected. The 
spongy mesophyll layer typically lies below the pali- 
sade mesophyll layer and is composed of many cells 
and intercellular air spaces as shown in Figure 8-35. It 
is here that the oxygen and carbon dioxide exchange 
takes place for photosynthesis and respiration. In the 
near-infrared part of the spectrum, healthy green vege- 
tation is generally characterized by high reflectance (40 
— 60%), high transmittance (40 — 60%) through the leaf 
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FIGURE 8-38 Hemispheric reflectance, transmittance, and absorption characteristics of Big Bluestem grass adaxial leaf sur- 
faces obtained using a laboratory spectroradiometer. The reflectance and transmittance curves are almost mirror images of 
one another throughout the visible and near-infrared portions of the electromagnetic spectrum. The blue and red chlorophyll 
in plants absorbs much of the incident energy in the visible portion of the spectrum (0.4 — 0.7 Km) (based on Walter-Shea and 
Biehl, 1990). Imaging spectrometers such as AVIRIS are capable of identifying small changes in the absorption and reflection 
characteristics of plants because the sensors often have channels that are only 10 nm apart, i.e., there could be 10 channels in 


the region from 0.6 to 0.7 Km (600 — 700 nm). 


onto underlying leaves, and relatively low absorptance 
(5-10%). Notice that a healthy green leaf’s reflectance 
and transmittance spectra throughout the visible and 
near-infrared portion of the spectrum are almost mir- 
ror images of one another, as shown in Figure 8-38 
(Walter-Shea and Biehl, 1990). 


The high diffuse reflectance of the near-infrared (0.7 — 
1.2 Km) energy from plant leaves is due to the internal 
scattering at the cell wall—air interfaces within the leaf 
(Gausmann et al., 1969; Peterson and Running, 1989). 
A well-known water vapor absorption band exists at 
0.92 — 0.98 Km; consequently, the optimum spectral re- 
gion for sensing in the near-infrared region is believed 
to be 0.74 — 0.90 Km (Tucker, 1979). 


The main reasons that healthy plant canopies reflect so 
much near-infrared energy are: 


* the leaf already reflects 40-60% of the incident near- 
infrared energy from the spongy mesophyll (Figure 
8-34), and 

* the remaining 45-50% of the energy penetrates (i.e., 
is transmitted) through the leaf and can be reflected 
once again by leaves below it. 


This is called / af addi we refl etane. For example, 
consider the reflectance and transmission characteris- 
tics of the hypothetical two-layer plant canopy shown 
in Figure 8-39. Assume that leaf 1 reflects 50% of the 
incident near-infrared energy back into the atmosphere 
and that the remaining 50% of the near-infrared energy 
is transmitted through leaf 1 onto leaf 2. The transmit- 
ted energy then falls on leaf 2 where 50% again is trans- 
mitted (25% of the original) and 50% is reflected. The 
reflected energy then passes back through leaf 1, which 
allows half of that energy (or 12.5% of the original) to 
be transmitted and half reflected. The resulting total 
energy exiting leaf 1 in this two-layer example is 62.5% 
of the incident energy. Therefore, the greater the num- 
ber of leaf layers in a healthy, mature canopy, theoreti- 
cally the greater the infrared reflectance. Conversely, if 
the canopy is only composed of a single, sparse leaf 
layer then the near-infrared reflectance will not be as 
great because the energy that is transmitted through 
the leaf layer may be absorbed by the ground cover be- 
neath the canopy. 


Changes in the near-infrared spectral properties of 
healthy green vegetation may provide information 
about plant senescence and/or stress. For example, 
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FIGURE 8-39 A hypothetical example of additive reflectance from a canopy with two leaf layers. Fifty percent of the inci- 
dent radiant flux, ® ;, to leaf 1 is reflected (Rj) and the other 50% is transmitted onto leaf 2 (T;). Fifty percent of the radiant 
flux incident to leaf 2 is transmitted through leaf 2 (T>); the other 50% is reflected toward the base of leaf 1 (R>). Fifty percent 
of the energy incident at the base of leaf 1 is transmitted through it (T3) while the remaining 50% (R3) is reflected toward leaf 
2 once again. At this point, an additional 12.5% (1/8) reflectance has been contributed by leaf 2, bringing the total reflected 
radiant flux to 62.5%. However, to be even more accurate, one would have to also take into account the amount of energy 
reflected from the base of leaf 1 (R3) onto leaf 2, and the amount reflected from leaf 2 (R4) and eventually transmitted 


through leaf 1 once again (Ts). This process would continue. 


consider the four leaves and their spectral reflectance 
characteristics shown in Figure 8-37. The photosynthe- 
sizing green leaf (a) exhibited strong chlorophyll ab- 
sorption in the blue and red wavelength regions, an 
understandable increase in green reflectance, and ap- 
proximately 76% reflectance in the near-infrared re- 
gion. After a certain point, near-infrared reflectance 
decreased as the leaves senesced (b—d). However, if the 
leaves were to dry out significantly during senescence, 
we would expect to see much higher reflectance in the 
near-infrared region (to be discussed shortly). 


Scientists have known since the 1960s that a direct rela- 
tionship exists between response in the near-infrared 
region and various biomass measurements. Conversely, 
it has been shown that an inverse relationship exists be- 
tween the response in the visible region, particularly 
the red, and plant biomass. The best way to appreciate 
this is to plot all of the pixels in a typical remote sens- 
ing scene in red and near-infrared reflectance space. 
For example, Figure 8-40a suggests where approxi- 
mately 10,000 pixels in a typical agricultural scene are 


located in red and near-infrared multispectral feature 
space (i.e., within the green area). Dry bare soil fields 
and wet bare soil fields in the scene would be located at 
opposite ends of the soil line. This means that a wet 
bare soil would have very low red and near-infrared re- 
flectance. Conversely, a dry bare soil area would prob- 
ably have high red and high near-infrared reflectance. 
As a vegetation canopy matures, it reflects more near- 
infrared energy while at the same time absorbing more 
red radiant flux for photosynthetic purposes. This 
causes the spectral reflectance of the pixel to move in a 
perpendicular direction away from the soil line. As bio- 
mass increases and as the plant canopy cover increases, 
the field’s location in the red and near-infrared spectral 
space moves farther away from the soil line. 


Figure 8-40b demonstrates how just one agricultural 
pixel might move about in the red and near-infrared 
spectral space during a typical growing season. If the 
field was prepared properly, it would probably be locat- 
ed in the moist bare soil region of the soil line with low 
red and low near-infrared reflectance at the beginning 
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FIG RE 8-40 a) The distribution of all the pixels in a scene in red and near-infrared multispectral feature space is found in 
the green shaded area. Wet and moist bare-soil fields are located along the soil line. The greater the biomass and/or crop 
canopy closure, the greater the near-infrared reflectance and the lower the red reflectance. This condition moves the pixel’s 
spectral location a perpendicular direction away from the soil line. b) The migration of a single veget ated agricultural pixel in 
red and near-infrared multispectral feature space during a growing season is shown. After the crop emerges, it departs from 
the soil line, eventually reaching complete canopy closure. After harvesting, the pixel will be found on the soil line, but per- 


haps in a drier soil condition. 


of the growing season. After the crop emerges from the 
ground, it would depart from the soil line, eventually 
reaching complete canopy closure. At this point, the 
reflected near-infrared radiant flux would be high and 
the red reflectance would be low. After harvesting, the 
pixel would probably be found once again on the soil 
line but perhaps in a drier condition. 


The relationship between red and near-infrared canopy 
reflectance has resulted in the development of remote 
sensing vegetation indices and biomass-estimating 
techniques that utilize multiple measurements in the 
visible and near-infrared region (e.g., Liu et al, 2012). 
The result is a linear combination that may be more 
highly correlated with biomass than either red or near- 
infrared measurement alone. 


Middle-Infrared Energy Interaction with Water 
in the Spongy Mesophyll 

Plants require water to grow. A leaf obtains water 
through the plant’s roots. The water travels from the 
roots, up the stem, and enters the leaf through the peti- 
ole. Veins carry water to the cells in the leaf. If a plant 
is watered so that it contains as much water as it can 
possibly hold at a given time, it is said to be fully tur- 
gid. Much of the water is found in the spongy meso- 


phyll portion of the plant. If we forget to water the 
plant or rainfall decreases, the plant will contain an 
amount of water that is less than it can potentially 
hold. This is called its relative turgidity. It would be 
useful to have a remote sensing instrument that was 
sensitive to how much water was actually in a plant 
leaf. Remote sensing in the middle-infrared, thermal 
infrared, and passive microwave portion of the electro- 
magnetic spectrum can provide such information to a 
limited extent. 


Water in the atmosphere creates five major absorption 
bands in the near-infrared through middle-infrared 
portions of the electromagnetic spectrum at 0.97, 1.19, 
1.45, 1.94, and 2.7 Km (Figure 8-34). The fundamental 
vibrational water-absorption band at 2.7 Xm is the 
strongest in this part of the spectrum (there is also one 
in the thermal infrared region at 6.27 Km). However, 
there is also a strong relationship between the reflec- 
tance in the middle-infrared region from 1.3 — 2.5 Km 
and the amount of water present in the leaves of a 
plant canopy. Water in plants absorbs incident energy 
between the absorption bands with increasing strength 
at longer wavelengths. In these middle-infrared wave- 
lengths, vegetation reflectance peaks occur at about 1.6 
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FIGURE 8-41 Reflectance response of a single magnolia leaf (Magnolia grandiflora) to decreased relative water content. As 
moisture content decreased, reflectance increased throughout the 0.4 to 2.5 Xm region. However, the greatest increase oc- 
curred in the middle-infrared region from 1.3 to 2.5 im (based on Carter, 1991). 


and 2.2 Xm, between the major atmospheric water ab- 
sorption bands (Figure 8-34). 


Water is a very good absorber of middle-infrared ener- 
gy, so generally, the greater the turgidity of the leaves, 
the lower the middle-infrared reflectance. Conversely, 
as the moisture content of leaves decreases, reflectance 
in the middle-infrared region increases substantially. 
As the amount of plant water in the intercellular air 
spaces decreases, this causes the incident middle-infra- 
red energy to be more intensely scattered at the inter- 
face of the intercellular walls resulting in much greater 
middle-infrared reflectance from the leaf. For example, 
consider the spectral reflectance of Magnolia leaf sam- 
ples at five different moisture conditions displayed over 
the wavelength interval from 0.4 — 2.5 Km (Figure 8- 
41). The middle-infrared wavelength intervals from 
about 1.5 — 1.8 Xm and from 2.1 — 2.3 Xm appear to be 
more sensitive to changes in the moisture content of 
the plants than the visible or near-infrared portions of 
the spectrum (1.e., the y-axis distance between the spec- 
tral reflectance curves is greater as the moisture con- 
tent decreases). Also note that substantive changes in 
the visible reflectance curves (0.4 — 0.7 Xm) did not be- 
gin to appear until the plant moisture in the leaves de- 
creased to about 50%. When the relative water content 
of the plant decreases to approximately 50%, almost 
any portion of the visible, near- and middle-infrared 
regions might provide some valuable spectral reflec- 
tance information. 


Reflectance in the middle-infrared region is inversely 
related to the absorptance of a layer of water approxi- 
mately 1 mm in depth (Gr ter, 1991). The degree to 
which incident solar energy in the middle-infrared re- 


gion is absorbed by vegetation is a function of the total 
amount of water present in the leaf and the leaf thick- 
ness. If proper choices of sensors and spectral bands 
are made, it is possible to monitor the relative turgidity 
in plants. 


Most optical remote sensing systems (except radar) are 
generally constrained to function in the wavelength in- 
tervals from 0.3 — 1.3, 1.5— 1.8, and 2.0 — 2.6 Xm due to 
the strong atmospheric water absorption bands at 1.45, 
1.94, and 2.7 Xm. Fortunately, as demonstrated in Fig- 
ure 8-34 there is a strong “carryover” sensitivity to wa- 
ter content in the 1.5 — 1.8 and 2.0 — 2.6 Xm regions 
adjacent to the major water absorption bands. This is 
the main reason that the Landsat Thematic Mapper (4 
and 5) and Landsat 7 Enhanced Thematic Mapper 
Plus (ETM*) were made sensitive to two bands in this 
region: band 5 (1.55 — 1.75 Mm) and band 7 (2.08 — 2.35 
Mim). The 1.55 — 1.75 Xm middle-infrared band has con- 
sistently demonstrated a sensitivity to canopy moisture 
content. For example, Pierce et al. (1990) found that 
this band and vegetation indices produced using it 
were correlated with canopy water stress in coniferous 
forests. 


Much of the water in a plant is lost via transpiration. 
Transpiration occurs as the Sun warms the water inside 
the leaf, causing some of the water to change its state 
to water vapor that escapes through the stomata. The 
following are several important functions that transpi- 
ration performs: 


¢ It cools the inside of the leaf because the escaping 
water vapor contains heat. 


¢ It keeps water flowing up from the roots, through 
the stem, to the leaves. 


¢ It ensures a steady supply of dissolved minerals 
from the soil. 


As molecules of water vapor at the top of the leaf in the 
tree are lost to transpiration, the entire column of wa- 
ter is pulled upward. Plants lose a considerable amount 
of water through transpiration each day. For example, 
a single corn plant can lose about 4 quarts (3.8 liters) 
of water on a very hot day. If the roots of the plant can- 
not replace this water, the leaves wilt, photosynthesis 
stops, and the plant dies. Thus, monitoring the mois- 
ture content of plant canopies, which is correlated with 
rates of transpiration, can provide valuable informa- 
tion on the health of a crop or stand of vegetation. 
Thermal infrared and passive microwave remote sens- 
ing have provided valuable plant canopy evapotranspi- 
ration information. 


The most practical application of plant moisture infor- 
mation is the regional assessment of crop water condi- 
tions for irrigation scheduling, stress assessment, and 
yield modeling for agriculture, rangeland, and forestry 
management. 


Remote Sensing—Derived Vegetation 
Indices 


Since the 1960s, scientists have extracted and modeled 
various vegetation biophysical variables using remotely 
sensed data. Much of this effort has involved the use of 
vegetation indices—dimensionless, radiometric mea- 
sures that indicate the relative abundance and activity 
of green vegetation, including leaf-area-index (LAI), 
percentage green cover, chlorophyll content, green bio- 
mass, and absorbed photosynthetically active radiation 
(APAR). A vegetation index should (Running et al., 
1994; Huete and Justice, 1999): 


* maximize sensitivity to plant biophysical parame- 
ters, preferably with a linear response in order that 
sensitivity be available for a wide range of vegetation 
conditions, and to facilitate validation and calibra- 
tion of the index; 


* normalize or model external effects such as Sun 
angle, viewing angle, and the atmosphere for consis- 
tent spatial and temporal comparisons; 


* normalize internal effects such as canopy back- 
ground variations, including topography (slope and 
aspect), soil variations, and differences in senesced 
or woody vegetation (nonphotosynthetic canopy 
components); 


* be coupled to some specific measurable biophysical 
parameter such as biomass, LAI, or APAR as part 
of the validation effort and quality control. 
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There are many vegetation indices. Good summaries 
about vegetation indices are found in Cheng et al. 
(2008), Galvao et al., (2009), Vina et al. (2011), Gray 
and Song (2012), Liu et al. (2012), and Wang et al. 
(2013). Many are functionally equivalent (redundant) 
in information content (Perry and Lautenschlager, 
1984), and some provide unique biophysical informa- 
tion. It is useful to review the historical development of 
some of the major indices and provide information 
about recent advances in index development. A few of 
the most widely adopted are summarized in Table 8-9. 


Simple Ratio—SR 

Many indices make use of the inverse relationship be- 
tween red and near-infrared reflectance associated with 
healthy green vegetation (Figure 8-42a). Cohen (1991) 
suggests that the first true vegetation index was the 
Simple Ratio (SR), which is the ratio of red reflected 
radiant flux (p ;¢q) to near-infrared radiant flux (p ,;,) 
as described in Birth and McVey (1968): 


= Pred 


SR (8.36) 


Phir 


The simple ratio provides valuable information about 
vegetation biomass or LAI (Schlerf et al., 2005; Pena- 
Barragan et al., 2011; Gray and Song, 2012). It is espe- 
cially sensitive to biomass and/or LAI variations in 
high-biomass vegetation such as forests (Huete et al., 
2002b). 


Normalized Difference Vegetation Index— 
NDVI 

Rouse et al. (1974) were some of the first to use what 
has become known as the Normalized Difference Vege- 
tation Index (NDVI): 


Pnir Pred 


NDVI = : 
Phir * Pred 


(8.37) 


The NDVI is functionally equivalent to the simple ra- 
tio; that is, there is no scatter in an SR vs. NDVI plot, 
and each SR value has a fixed NDVI value. When we 
plot the mean NDVI and SR values for various bi- 
omes, we find that the NDVI approximates a nonlinear 
transform of the simple ratio (Figure 8-42b) (Huete et 
al., 2002b). The NDVI is an important vegetation in- 
dex because: 


¢ Seasonal and inter-annual changes in vegetation 
growth and activity can be monitored. 


¢ The ratioing reduces many forms of multiplicative 
noise (Sun illumination differences, cloud shadows, 
some atmospheric attenuation, some topographic 
variations) present in multiple bands of multiple- 
date imagery. 
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Physiological Basis for Developing Vegetation Indices 


Carotenoid and 
chlorophyll 
absorption 


Foliar 
reflectance 


Liquid water 
absorption 


Chlorophyll 
absorption 
Transition 
Transition 


| 


Reflectance, (%) 


e 


*.. Reduced chlorophyll absorption 
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FIGURE 8-42 a) The physiological basis for developing vegetation indices. Typical spectral reflectance characteristics for 
healthy green grass and bare dry soil for the wavelength interval from 250 - 2,500 nm. b) The NDVI is a normalized ratio of 
the near-infrared and red bands. The NDVI is functionally equi valent to and is a nonlinear transform of the simple ratio 
(based on Huete et al., 2002b). 


Following are disadvantages of the NDVI (Huete et al., 
2002a; Wang et al., 2004): 


¢ The ratio-based index is nonlinear and can be influ- 
enced by additive noise effects such as atmospheric 
path radiance (Chapter 6). 


¢ NDVI is usually highly correlated with vegetation 
leaf-area-index (LAI). However, the relationship 
may not be as strong during periods of maximum 
LAI, apparently due to the saturation of NDVI 
when LAI is very high (Wang et al., 2004). For 
example, Figure 8-42b reveals that the NDVI 
dynamic range is stretched in favor of low-biomass 
conditions and compressed in high-biomass, for- 
ested regions. The opposite is true for the Simple 
Ratio, in which most of the dynamic range encom- 
passes the high-biomass forests with little variation 
reserved for the lower-biomass regions (grassland, 
semi-arid, and arid biomes). 


¢ It is very sensitive to canopy background variations 
(e.g., soil visible through the canopy). NDVI values 
are particularly high when darker-canopy back- 
grounds exist. 


Scientists continue to use the NDVI heavily (e.g. Gal- 
vao et al., 2005, 2009; Neigh et al., 2008; Mand et al., 
2010; Sonnenschein et al., 2011; Vina et al., 2011, Liu 
et al., 2012; Liet al., 2014). In fact, two of the standard 
MODIS land products are sixteen-day composite 
NDVI datasets of the world at a spatial resolution of 
500 m and | km (Huete et al., 2002a). NDVI images 
derived from MODIS data obtained in February and 
October, 2012, are shown in Figure 8-43. Note the 
“sreening-up” of the terrain in the Northern hemi- 
sphere, especially in Canada, Europe, and Russia. Also 
note the greening-up of the Sahel region of Africa just 
below the Sahara Desert. Geospatial biophysical infor- 
mation such as this is used in many biophysical models. 


The NDVI transform also performs well on high spa- 
tial resolution imagery that has red and near-infrared 
bands. For example, consider the NDVI image extract- 
ed from the color-infrared 1 x 1 ft. spatial resolution 
digital frame camera photography shown in Figure 8- 
44. The single-band NDVI image is displayed in Figure 
8-44e. The brighter the shade of gray in the NDVI im- 
age, the greater the amount of green biomass. Note 
that the water in the swimming pools, the street con- 
crete, and building rooftops all have low NDVI values 
(i.e., they are dark). High NDVI values associated with 
vegetation are very easy to locate in the color density- 
sliced image shown in Figure 8-44f. 


Kauth-Thomas Tasseled Cap Transformation 

Kauth and Thomas (1976) developed an orthogonal 
transformation of the original Landsat MSS data 
space to a new four-dimensional feature space. It was 
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called the tasseled cap or Kauth-Thomas transforma- 
tion. It created four new axes: the soil brightness index 
(B), greenness vegetation index (G), yellow stuff index 
(Y), and none-such (NV). The names attached to the 
new axes indicate the characteristics the indices were 
intended to measure. The coefficients are (Kauth et al., 
1979): 


B = 0.332MSSI + 0.603 MSS2 + 0.675MSS3 + 0.262MSS4_— (8.38) 
G = —0.283MSS1 — 0.660MSS2 + 0.577MSS3 + 0.388MSS4 (8.39) 
Y = —0.899MSS1 + 0.428MSS2 + 0.076MSS3 — 0.041MSS4_ (8.40) 


N = —0.016MSS/ + 0.131 MSS2 — 0.452MSS3 + 0.882MSS4. (8.41) 


Crist and Kauth (1986) determined the visible, near-in- 
frared, and middle-infrared coefficients for transform- 


ing Landsat Thematic Mapper imagery into 
brightness, greenness, and wetness variables: 
B = 0.2909TM1 + 0.2493 TM2 + 0.48067M3 + (8.42) 
0.5568 7M4 + 0.4438 7M5 + 0.17067M7 
G = —0.2728TM1 — 0.2174TM2 — 0.5508 TM3 + (8.43) 
0.7221 TM4 + 0.0733 TM5 — 0.1648 7M7 
W = 0.14467M1 + 0.1761 TM2 + 0.3322TM3 + (8.44) 


0.3396 TM4 — 0.62107MS5 — 0.41867M7. 


It is called the tasseled cap transformation because of 
its unique cap shape (Yarbrough et al., 2012) (Figures 
8-45a—c). Crist and Cicone (1984) identified a third 
component that is related to soil features, including 
moisture status (Figure 8-45d). Thus, an important 
source of soil information is available through the in- 
clusion of the middle-infrared bands of the Thematic 
Mapper (Price et al., 2002). 


The 1982 Charleston, SC, Landsat TM scene was 
transformed into brightness, greenness, and wetness 
(moisture) content images (Figure 8-46) based on the 
use of the TM tasseled cap coefficients (Equations 8-42 
through 8-44). Urbanized areas are particularly evi- 
dent in the brightness image. The greater the biomass, 
the brighter the pixel value in the greenness image. The 
wetness image provides subtle information concerning 
the moisture status of the wetland environment. As ex- 
pected, the greater the moisture content, the brighter 
the response. A color composite of the dataset is 
shown in Figure 8-46d (RGB = brightness, greenness, 
and wetness). 


The tasseled cap transformation is a global vegetation 
index. Theoretically, it can be used anywhere in the 
world to disaggregate the amount of soil brightness, 
vegetation, and moisture content in individual pixels in 
Landsat MSS, TM, ETM", Landsat 8 or other types of 
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Selected MODIS Normalized Difference Vegetation Index (NDVI) Composite Images in 2012 


a. NDVI composite derived from MODIS images 
collected from February 1, 2012, to March 1, 2012. 


b. NDVI composite derived from MODIS images 
collected from October 1, 2012, to November 1, 2012. 


FIGU RE 8-43 a) NDVI composite image derived from MODIS data acquired from February 1, 2012, to March 1, 2012. The 
darker the green, the higher the NDVI value and the greater the leaf-area-index (LAI) and biomass. b). NDVI composite 
image derived from MODIS data acquired from October 1, 2012, to November 1, 2012 (imagery courtesy of NASA Earth 
Observations - NEO). 
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Normalized Difference Vegetation Index (NDVI) Image Extracted 


a. Color composite (RGB = NIR, red, green). 


' 


en 


e. NDVI derived from red and near-infrared bands. 


f. Color density-sliced NDVI. 


FIGURE 8-44 An example of the NDVI transformation applied to color-infrared 1 1 ft. spatial resolution digital frame cam- 
era imagery (original aerial photography courtesy of John Copple, Sanborn Map Company). a) Original image. b—d). Green, 
red, and near-infrared bands. e) NDVI image. f) Density-sliced NDVI image. 


multispectral data. Practically, however, it is better to 
compute the coefficients based on local conditions if 
possible. Jackson (1983) provided a computer program 
for this purpose. 


The Kauth-Thomas tasseled cap transformation con- 
tinues to be widely used (e.g., Powell et al., 2010; Son- 


nenschein et al., 2011; Pflugmacher et al., 2012; 
Yarbrough et al., 2012, 2014; Ali Baig et al., 2014). 
Huang et al. (2002) developed tasseled cap coefficients 
for use with Landsat 7 ETM” at-satellite reflectance 
data (Table 8-10). These coefficients are most appropri- 
ate for regional applications where atmospheric correc- 
tion is not feasible. Yarbrough et al. (2012) developed a 
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TABLE 8-9 Selected remote sensing vegetation and other indices. 


Vegetation Index 


Equation 


Selected References 


Simple Ratio (SR) 


Birth and McVey, 1968 
Colombo et al., 2003 
Schlerf et al., 2005 
Boer et al., 2008 
Pena-Barragan, 2011 
Vina et al., 2011 

Gray and Song, 2012 


Normalized Difference Vegetation 
Index (NDVI) 


Pnir BS Pred 


Rouse et al., 1974 
Deering et al., 1975 
Huete et al., 2002a 
Colombo et al., 2003 
Schlerf et al., 2005 
Houborg et al., 2007 
Boer et al., 2008 

Cheng et al., 2008 
Neigh et al., 2008 
Mand et al., 2010 
Sonnenschein et al., 2011 
Pena-Barragan, 2011 
Purkis and Klemas, 2011 
Sims et al., 2011 

Vina et al., 2011 

Gray and Song, 2012 
Liu et al., 2012 

Li et al., 2014 


Kauth-Thomas Transformation 
Brightness 

Greenness 

Yellow stuff 

None-such 


Brightness 
Greenness 
Wetness 


Landsat Multispectral Scanner (MSS) 

B = 0.332MSS1 + 0.603 MSS2 + 0.675 MSS3 + 0.262 MSS4 
G = — 0.283 MSS1 — 0.660MSS2 + 0.577MSS3 + 0.388 MSS4 
Y = —0.899MSS1 + 0.428 MSS2 + 0.076 MSS3 — 0.041 MSS4 
N = —0.016MSS/ + 0.131MSS2 — 0.452 MSS3 + 0.882 MSS4 


Landsat Thematic Mapper (TM) 
B = 0.2909TM1 + 0.2493 TM2 + 0.4806 7M3 + 
0.5568 T7M4 + 0.4438 7M5 + 0.1706TM7 
= —0.2728TM1 — 0.2174TM2 — 0.5508 TM3 + 
0.7221 7M4 + 0.0733 TM5 — 0.1648 TM7 
W = 0.14467M1 + 0.1761 7M2 + 0.33227M3 + 
0.3396 7M4 — 0.6210 7M5 — 0.41867M7 


Kauth and Thomas, 
1976 
Kauth et al., 1979 


Crist and Kauth, 1986 
Price et al., 2002 

Rogan et al., 2002 

Jin and Sader, 2005 
Schlerf et al., 2005 
Powell et al., 2010 
Sonnenschein et al., 2011 
Pflugmacher et al., 2012 
Yarbrough et al., 2012 
Ali Baig et al., 2014 
Yarbrough, 2014 


Normalized Difference Moisture or Wa- 
ter Index (NDMI or NDW/I) 


(Dnie Pawir) 


NDMI or NDWI = 
(Pai * Psspin? 


Hardisky et al., 1983 
Gao, 1996; 

Jackson et al., 2004 
Galvao et al., 2005 
Jin and Sader, 2005 
Houborg et al., 2007 
Verbesselt et al., 2007 
Cheng et al., 2008 
Gray and Song, 2012 
Wang et al., 2013 


Perpendicular Vegetation Index (PVI) 


PVI = osssussa- 0.149MS52)- + (0.355MSS2— 0.852MSS4) 


| — (NIR=aRed~b) 


2 
lt+a 


PV 


Richardson and Wie- 
gand, 1977 


Colombo et al., 2003 
Guyon et al., 2011 


Leaf Rel ative Water Cont ent Index 
(LWCl) 


LWCI = 
log [1 — (NR py, ~ Mid IR pays, )] 


—log[1 — (NIR pyy4— Mid IR pys)] 


Hunt et al., 1987 
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Vegetation Index 


Equation 


Selected References 


Soil Adjusted Vegetation Index (SAVI) 


(Pnir — Pred) 


Huete, 1988 


(Pain + 0.5 P2.1um) 


and Modified SAVI (MSAVI) SAVI = ——————-(1 + L) Huete and Liu, 1994 
(Diet Pregt )) Al oe 1994 
Colombo et al., 2003 
Sonnenschein et al., 2011 
Atmospherically Resistant Vegetation G5 Dig Kaufman and Tanre, 
noe an a aseray hes and Liu, 1994 
Prir” Prb Colombo et al., 2003 
Soil and Atmospherically Resistant ee Huete and Liu, 1994; 
Vegetation Index (SARVI) SARVI = ee Running et al., 1994 
nil - Prb +L 
Enhanced Vegetation Index (EVI) Pye Pyag Huete et al., 1997 
EVI = CT g to Cloe EL ais ral 4 Huete ene tie 1999 
eel i + Huete et al., 2002a 
Prin 1Pred 2Polue Colombo et al., 2003 
TBRS, 2003 
Houborg et al., 2007 
Aloe et al., 2007 
Cheng, 2008 
Jiang et al., 2008 
oe Sims et al., 2011 
EVI2 = 5.5 Pnir Pred) _ Gray and Song, 2012 
(Oa 24D od +1) Liu et al., 2012 
Wagle et al., 2014 
Aerosol Free Vegetation Index (AFRI) (P nip — 0.66p, 6 cao Karnieli et al., 2001 
AFRI, 6um = one : 
; CP = 66P1 6m) 
AFRI _ (Prir~ 9-5P 2.1m) 
2.1um 


Triangular Vegetation Index (TVI) 


1 
TIVE 5 120(B i= Bispuen)) 200 ga Pareen) 


Broge and Leblanc, 
2000 

Pena-Barragan et al., 
2011 


Reduced Simple Ratio (RSR) 


RSR = Pswir  Pswirmin 


Prir i = 
Pred P swirmax = Pswirmin 


Chen et al., 2002 
Gray and Song, 2012 


Ratio TCARI /OSAVI 


- P700 
TCARI = 3 {Pro —P670) — 9.2(P 799 - Psso)| ) 


P 670 
(1+ 0.16) (Peo - P6790) 


Kim et al., 1994 
Rondeaux et al., 1996 
Daughtry et al., 2000 
Haboudane et al., 2002 
Pena-Barragan, 2011 


OSAVI= 
(Pgoo * Pe70 * 9-16) 
TCARI 
OSAVI 
Visible Atmospherically Resistant a Gitelson et al., 2002 
Index (VARI) VARI, oon = green !re 
= Pereen - Pred~ Phlue 
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TABLE 8-9 Selected remote sensing vegetation and other indices. 


Vegetation Index 


Equation 


Selected References 


Normalized Difference Built-up Index 
(NDBI) 


MidIR py5—-NIR bya 


NDBI = — 
MidIR yrs + NIR rap 


built-up... = NDBI-NDVI 


Zha et al., 2003 


Vegetation Adjusted Nighttime Lights 
(NTL) Urban Index (VANUI) 


VANUI = (1—NDVI) x NTL 


Zhang et al., 2013 


Red-edge Position 


P(red edge) — °(700nm) 
P(740nm) — °(700nm) 


REP = 700 + 40 


Clevers, 1994 
Dawson and Curran, 
1998 

Baranoski, 2005 


where 
= Pe 670nm) ¥ P(780nm) 

P(red edge) — 2 
Photochemical Reflectance Index (PRI) = Mand et al. 2010 

PRI = (531 -P 570) Sims et al., 2011 

(P531 * P5709) 

NDVI and Cellulose Absorption Index —_ G h t al. 2009 
(CAI) to Quant ify Fractional Cover of NDVI = Pnir Pred ae 
Photosynthetic Vegetation, Non- Op 
photosynthetic Vegetation and Bare nir red 
Soil 

and 


CAI = [0.5 x (P79 + P29) — Po 4] x 10 


Wang et al., 2013 


MERIS Terrestrial Chlorophyll Index 
(MTCl) 


MT cr = Pband 107 Pband 9 _ P753.75~ P708.75 
Phand 9~ Phand 8 


Dash et al., 2010 


P708.75 ~ P681.25 


Normalized Burn Ratio (NBR) ees ee Brewer et al., 2005 
NBR = 222230 Boer et al., 2008 
Pnir a Pswir 
ANBR = NBR 6 —fire NBR 651 — fire 


Kauth-Thomas transform workflow that includes im- 
age screening methods to determine if scenes are ap- 
propriate for application of the transformation. 
ERDAS Imagine and ENVI software provide Kauth- 
Thomas transformation algorithms. 


Normalized Difference Moisture or Water 
Index—NDMI or NDWI 

As previously discussed, information about vegetation 
water content has widespread utility in agriculture, for- 
estry, and hydrology (Galvao et al., 2005). Hardisky et 
al. (1983) and Gao (1996) found that the Normalized 
Difference Moisture or Water Index (NDMI or NDWI) 


based on Landsat TM near- and middle-infrared 
bands, 


(Pa Pewiy) 
NDMI or NDWI = —“1—**" 


(8.45) 
coe = P sie) 


was highly correlated with canopy water content and 
more closely tracked changes in plant biomass and wa- 
ter stress than did the NDVI. Jackson et al. (2004) 
found that the NDVI saturated while the NDWI con- 
tinued to document changes in corn and soybean vege- 
tation water content. Jin and Sader (2005) compared 
the NDMI and the tasseled cap wetness transforma- 
tion for detecting forest disturbance. Verbesselt et al. 
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Characteristics of the Kauth-Thomas Tasseled Cap Transformation 


Greenness 


Brightness 


a. Crop development in the tasseled cap 
brightness-greenness transformation. 


Greenness 


Brightness 


c. Actual plot of brightness and greenness 
values for an agricultural area. The shape 
of the distribution looks like a cap. 


Greenness 


b. 


Concrete, 
dry bare soil 


Os Tia 
water 


Brightness 


Location of land cover when plotted 


in the brightness-greenness spectral space. 


Third 


Clear 
water Turbid 

water ; 
Brightness 


Wet Dry 
soul bare Concrete 
soil 


d. Approximate direction of moisture 


variation in the plane of soils. The arrow 


points in the direction of less moisture. 


FIGURE 8-45 General characteristics of the Kauth-Thomas tasseled cap transformation (based on Crist and Cicone, 1984; 
Crist and Kauth, 1986). The tasseled cap transformation is applied correctly if the tasseled cap’s base is parallel with the 


brightness axis. 


TABLE 8-10 Tasseled cap coefficients for use with Landsat 
7 ETM? data (Huang et al., 2002). 


Index ™1 | TM2 | TM3 | TM4 | TMS | TM7 
Brightness .3561 3972 -3904 -6966 .2286 -1596 
Greenness -.334 -.354 -.456 -6966 -.024 -.263 
Wetness -2626 .2141 .0926 .0656 -.763 -.539 
Fourth -0805 -.050 -1950 -.133 .5752 -.777 
Fifth —.725 -.020 -6683 .0631 -.149 -.027 
Sixth .400 -.817 .3832 .0602 -.109 .0985 

(2007) used the NDWI to monitor herbaceous fuel 
moisture based on SPOT VEGETATION data. Cheng 


et al. (2008) applied the NDWI to MODIS data and 
compared the results to AVIRIS-derived canopy equiv- 
alent water thickness (EWT). 


Perpendicular Vegetation Index—PVI 


Richardson and Wiegand (1977) were instrumental in 
the development of the perpendicular distance to the 


“soil line” as an indicator of plant development. The 
“soil line,” which is a two-dimensional analog of the 
Kauth-Thomas soil brightness index, was estimated by 
linear regression. The Perpendicular Vegetation Index 
(PVI) based on MSS band 4 data was 


PVI = dtoassmss4 —0.149MSS2)° + (0.355MSS2—0.852MSS4)" (8.46) 


Guyon et al. (2011) used a more generic PVI to moni- 
tor leaf phenology in deciduous broadleaf forests 
where: 


(NIR — aRed-—b) 


2 
lt+a 


PVI = (8.47) 


and a and b are the slope and the intercept of the soil 
line, respectively. 


Leaf Water Content Index—LWCI 
Hunt et al. (1987) developed the Leaf Water Content 
Index (LWCI) to assess water stress in leaves: 


log[1 — (NIR pypq— Mid IR pyyp5)] 


LAW CD wae 
slog[1 —(NIRpyg —MidIRpys )| 


(8.48) 
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Kauth-Thomas (Tasseled Cap) Brightness, Greenness, and Wetness Transformation 


of Landsat Thematic Imagery of Charleston, SC, Collected on November 9, 1982 
_ wr . 


c. Wetness. 


d. Color-composite (RGB = Brightness, 
Greenness, and Wetness). 


FIGURE 8-46 Brightness, greenness, and wetness images derived by applying Kauth-Thomas tasseled cap transformation co- 
efficients to November 9, 1982, Charleston, SC, Thematic Mapper data (the thermal band was not used). Original imagery 


courtesy of NASA. 


where ft represents reflectance in the specified bands 
when leaves are at their maximum relative water con- 
tent (fully turgid; RWC) defined as: 


RWC = field weig ht — oven dry weig ht x 100 (8.49) 
turgid weight — oven dry weight 


Soil Adjusted Vegetation Index—SAVI 

The utility of the NDVI and related indices for satellite 
and airborne assessment of the Earth’s vegetation cov- 
er has been demonstrated for almost three decades. 
The time series analysis of seasonal NDVI data has 
provided a method of estimating net primary produc- 
tion over varying biome types, of monitoring pheno- 
logical patterns of the Earth’s vegetated surface, and of 
assessing the length of the growing season and dry- 
down periods (Huete and Liu, 1994; Ramsey et al., 


1995). For example, global vegetation analysis was ini- 
tially based on linearly regressing NDVI values (de- 
rived from AVHRR, Landsat MSS, Landsat TM, and 
SPOT HRV data) with in situ measurements of LAI, 
APAR, percent cover, and/or biomass. This empirical 
approach revolutionized global science land-cover bio- 
physical analysis in just one decade (Running et al., 
1994). Unfortunately, studies have found that the em- 
pirically derived NDVI products can be unstable, vary- 
ing with soil color and moisture conditions, 
bidirectional reflectance distribution function (BRDF) 
effects, atmospheric conditions, and the presence of 
dead material in the canopy itself (Qi et al., 1995). For 
example, Goward et al. (1991) found errors of +50% in 
NDVI data used for global vegetation studies derived 
from the NOAA Global Vegetation Index product. 
What is needed are globally accurate NDVI-related 
products that do not need to be calibrated by in situ 


measurement within each geographic area yet will re- 
main constant under changing atmospheric and soil 
background conditions (Huete and Justice, 1999). 


Therefore, emphasis has been given to the development 
of improved vegetation indices that take advantage of 
calibrated sensor systems such as the Moderate Reso- 
lution Imaging Spectrometer (MODIS) (Running et 
al., 1994). Although the NDVI has been shown to be 
useful in estimating vegetation properties, many im- 
portant external and internal influences restrict its 
global utility. The improved indices typically incorpo- 
rate a soil background and/or atmospheric adjustment 
factor. 


The Soil Adjusted Vegetation Index (SAV]I) is 


CPingy = Pog) 


SAVI = 
Ce Pred +L) 


(22) (8.50) 


where L is a canopy background adjustment factor 
that accounts for differential red and near-infrared ex- 
tinction through the canopy (Huete, 1988; Huete et al., 
1992; Karnieli, et al., 2001). The value of L depends on 
the proportional vegetation cover as well as the vegeta- 
tion density. For very sparse vegetation or bare soils, L 
approximates one, whereas L converges to zero in 
densely vegetated areas (in this case the SAVI and 
NDVI become equal) (Huete and Liu, 1994; Sonnen- 
schein et al., 2011). The utility of SAVI for minimizing 
the soil “noise” inherent in the NDVI has been corrob- 
orated in many studies. Qi et al. (1995) developed a 
modified SAVI, called MSAVI, that uses an iterative, 
continuous L function to optimize soil adjustment and 
increase the dynamic range of SAVI. 


Atmospherically Resistant Vegetation Index— 
ARVI 

SAVI was made much less sensitive to atmospheric ef- 
fects by normalizing the radiance in the blue, red, and 
near-infrared bands. This became the Atmospherically 
Resistant Vegetation Index (ARVI): 


(8.51) 


ARVI = a 


Phir t Pro 


where 


Pro ~ Prag Oye Pred * 


The technique requires prior correction for molecular 
scattering and ozone absorption of the blue, red, and 
near-infrared remote sensor data. ARVI uses the differ- 
ence in the radiance between the blue channel and the 
red channel to correct the radiance in the red channel 
and thus reduce atmospheric effects. Unless the aerosol 
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model is known a priori, gamma (y ) is normally equal 
to 1.0 to minimize atmospheric effects. Kaufman and 
Tanre (1992) provide guidelines about where different 
gammas might be used over continental, maritime, 
desert (e.g., the Sahel in Africa), or heavily vegetated 
areas. Colombo et al. (2003) used ARVI and other VIs 
with high spatial resolution IKONOS imagery and tex- 
ture measures to extract leaf-area-index information. 


Soil and Atmospherically Resistant Vegetation 
Index—SARVI 

Huete and Liu (1994) integrated the L function from 
SAVI and the blue-band normalization in ARVI to de- 
rive a Soil and Atmospherically Resistant Vegetation In- 
dex (SARVI) that corrects for both soil and 
atmospheric noise, as would a modified SARVI 
(MSARVI): 


Pie Py 
SARI =—22 (8.52) 
Pnir t Pry TE 
and 
2 
20, +1-[[ep +1) -Y¥(P,..- Pp) 
MSARVI = 2Pnir* 1 || 2Pnir* Y ~1Pnir“ Prod}. 5 


2 


Huete and Liu (1994) performed a sensitivity analysis 
on the original NDVI and improved vegetation indices 
(SAVI, ARVI, SARVI, MSARVI) and drew the follow- 
ing conclusions: 


¢ If there were a total atmospheric correction, then 
there would mainly be “soil noise,” and the SAVI 
and MSARVI would be the best equations to use, 
and the NDVI and ARVI would be the worst. 


¢ If there were a partial atmospheric correction to 
remove the Rayleigh and ozone components, then 
the best vegetation indices would be the SARVI and 
MSARVI, with the NDVI and ARVI being the 
worst. 


e If there were no atmospheric correction at all (i.e., 
no Rayleigh, ozone, or aerosol correction), the 
SARVI would become slightly worse but still would 
have the least overall noise. The NDVI and ARVI 
would have the most noise and error. 


Aerosol Free Vegetation Index—AFRI 
Karnieli et al. (2001) found that under clear sky condi- 
tions the spectral bands centered on 1.6 and 2.1 Xm are 
highly correlated with visible spectral bands centered 
on blue (0.469 Xm), green (0.555 Km), and red (0.645 
kim). Empirical linear relationships such as Po 469 ia 
9.2509 tum Po.5554um = 0.33) tum 29d PQ 645m = 
-66P1 6m Were found to be statistically significant. 
Therefore, based on these and other relationships, two 
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different Aerosol Free Vegetation Indices (AFRI) were 
developed: 


(Pnir = 0.669 6 fi) 


AFRI 5 ee OU and. (654) 
ia (Pnir + 9-667) 64m) 
(go 0:2 D>: jin) 
_ \Fnir Lum 
AFR jim = oe (8.55) 


CP sii + 0.5 P21 um) 


Under clear sky conditions, the AFRIs (especially 
AFRI) jm) result in values very similar to NDVI. 
However, if the atmospheric column contains smoke or 
sulfates then the AFRIs are superior to NDVIs. This is 
because the electromagnetic energy centered at 1.6 and 
2.1 Xm is able to penetrate the atmospheric column 
better than red wavelength energy used in the NDVI. 
Therefore, the AFRIs have a major application in as- 
sessing vegetation in the presence of smoke, anthropo- 
genic pollution, and volcanic plumes. Limited success 
is expected in the case of dust due to presence of larger 
particles that are similar in size to the wavelength, and 
therefore not transparent at 2.1 Xm (Kaufman et al., 
2000). The AFRIs can be implemented using any sen- 
sor that incorporates bands centered on 1.6 and/or 2.1 
Xm such as the Landsat TM, and ETM", Landsat 8, 
MODIS, ASTER, the Japanese Earth Resource Satel- 
lite-Optical System (JERS-OPS), SPOT 4-Vegetation, 
and IRS-1C/D. 


Enhanced Vegetation Index—EVI 

The MODIS Land Discipline Group developed the 
Enhanced Vegetation Index (EVI) for use with MODIS 
data (Huete et al., 1997; Jiang et al., 2008): 


Phir Pred 


EVI = G——————__—_———_ . 
Phir ‘i Cy Pred~ CoP bine +L 


(8.56) 


where nir, red, and blue are atmospherically corrected 
or partially atmospherically-corrected (Rayleigh and 
ozone absorption) surface reflectances. The EVI is a 
modified NDVI with a soil adjustment factor, L, and 
two coefficients, C; and C>, which describe the use of 
the blue band in correction of the red band for atmo- 
spheric aerosol scattering. The coefficients, C;, Cp, and 
L, are empirically determined as 6.0, 7.5, and 1.0, re- 
spectively. Gis a gain factor set to 2.5. This algorithm 
has improved sensitivity to high-biomass regions and 
has improved vegetation monitoring through a decou- 
pling of the canopy background signal and a reduction 
in atmospheric influences (Huete et al., 1997; Huete 
and Justice, 1999; Huete et al., 2002; Wagle et al., 
2014). A comparison of MODIS-derived NDVI values 
and MODIS-derived EVI values for dense vegetation 
is shown in Figure 8-47 (Didan, 2002). The EVI has 


improved sensitivity to high biomass. Wardlow et al. 
(2007) found that as NDVI began to approach an as- 
ymptotic level at the peak of the growing season over 
cropland, EVI exhibited more sensitivity during this 
growth stage. 


Two MODIS vegetation indices, the MODIS NDVI 
and the MODIS EVI, are produced at 1 x 1 km and 
500 x 500 m resolutions (and occasionally at 250 x 250 
m) using 16-day compositing periods. A MODIS EVI 
map of the Earth obtained starting on day 193 of 2003 
is shown in Figure 8-48. 


EVI2 is a two-band version of EVI. It was developed 
for use with remote sensing systems that do not have a 
blue band (Jiang et al., 2008; Liu et al., 2012): 


Oia Pred) 


EVI2 = 2.5———_——_.. 
(Print 24D p64 71) 


(8.57) 


Jiang et al. (2008) suggest that the differences between 
EVI and EVI? are insignificant over a large sample of 
snow/ice-free land cover types, phenologies, and scales 
when atmospheric influences are minimal. 


Triangular Vegetation Index—TVI 

Broge and Leblanc (2000) developed a Triangular Veg- 
etation Index (TVI), which describes the radiative ener- 
gy absorbed by pigments as a function of the relative 
difference between red and near-infrared reflectance in 
conjunction with the magnitude of reflectance in the 
green region, where the light absorption by chlorophyll 
aand b is relatively insignificant. The TVI index is cal- 
culated as the area of the triangle defined by the green 
peak, the chlorophyll absorption minimum, and the 
near-infrared shoulder in spectral space. It is based on 
the fact that both chlorophyll absorption causing a de- 
crease of red reflectance and leaf tissue abundance 
causing increased near-infrared reflectance will in- 
crease the total area of the triangle. The TVI index en- 
compasses the area spanned by the triangle ABC with 
the coordinates given in spectral space: 


)) — 200(p ) (8.58) 


nir Pgreen red Pgreen 


TVI = 1 U0(p 
2 
where P green» Pred» and Pp ,;, are the reflectances cen- 
tered at 0.55 Km, 0.67 Xm, and 0.75 Xm, respectively. 
Pena-Barragan et al. (2011) applied the index to AS- 
TER optical data for crop identification. 


Reduced Simple Ratio—RSR 

Chen et al. (2002) modified the simple ratio algorithm 
to include information from the short-wavelength in- 
frared (SWIR) band found in the SPOT VEGETA- 
TION sensor. They used the sensor to map the spatial 
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MODIS Enhanced Vegetation Index versus 
Normalized Difference Vegetation Index over Dense Vegetation 
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FIGJ RE 8-47 A comparison of the sensitivity of the MODIS-derived Normalized Difference Vegetation Index (NDVI) values 
and MODIS Enhanced Vegetation Index (EVI) values for dense vegetation (based on Didan, 2002). 


distribution of leaf-area-index in Canada. The Reduced 
Simple Ratio (RSR) is: 


P,; 
rr = Bae (1 


Pswir Pswirmin | (8.59) 
Pred 


, + a 
Pswirmax | Pswirmi 


where P syirmin 804 P sywirmax are the minimum and 
maximum SWIR reflectance found in each image and 
defined as the 1 percent mintmum and maximum cut- 
off points in the histograms of SWIR reflectance in a 
scene. The major advantages of RSR over SR are a) 
the difference between land-cover types is much re- 
duced so that the accuracy for leaf-area-index retrieval 
for mixed cover types can be improved or a single LAI 
algorithm can be developed without resorting to coreg- 
istered land-cover maps as the first approximation, and 
b) the background (understory, litter, and soil) influ- 
ence is suppressed using RSR because the SWIR band 
is most sensitive to the amount of vegetation contain- 
ing liquid water in the background (Chen et al., 2002). 
Gray and Song (2012) used the RSR in their mapping 
of leaf-area-index using spatial, spectral and temporal 
data from multiple sensors. 


Chlorophyll Absorption in Reflectance Index— 
CARI; Modified Chlorophyll Absorption in 
Reflectance Index—MTCARI; Optimized Soil- 
Adjusted Vegetation Index—OSAVI; Ratio 
TCARI/OSAVI 

Many scientists are interested in the amount of chloro- 
phyll in vegetation (e.g., Daughtry et al., 2000). Kim et 
al. (1994) developed the Chlorophyll Absorption in Re- 
flectance Index (CARI). It was modified to become the 
Transformed Chlorophyll Absorption in Reflectance In- 
dex (TCARI): 


: p 
TCARI = 3) (P7990 - P670) — 0.20P 90” Paso ] (8.60) 
: 


It uses bands corresponding to the minimum absorp- 
tion of the photosynthetic pigments, centered at 550 
and 700 nm, in conjunction with the chlorophyll a 
maximum absorption band, around 670 nm. The 
choice of 700 nm is due to its location at the boundary 
between the region where vegetation reflectance is 
dominated by pigment absorption and the beginning 
of the red edge portion where vegetation structural 
characteristics (i.e., the spongy mesophyll) have more 
influence on reflectance (Kim et al., 1994). 
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MODIS Enhanced Vegetation Index Map of the World 


FIGURE 8-48 MODIS Enhanced Vegetation Index (EVI) map of the world obtained over a 16-day period beginning on day 
193 of 2003. The greener the area, the greater the amount of biomass (courtesy Terrestrial Biophysics and Remote Sensing 


MODIS Team, University of Arizona and NASA). 


Unfortunately, TCARI is still sensitive to the underly- 
ing soil reflectance properties, particularly for vegeta- 
tion with a low leaf-area-index. Therefore, Daughtry et 
al. (2000) proposed that TCARI be combined with a 
generic soil line vegetation index like the Optimized 
Soil-Adjusted Vegetation Index (OSAVI) (Rondeaux et 
al., 1996): 


_ C+ 0.16)(Pgo9 ~ P670) 


OSAVI = (8.61) 
(Pgoo * P6790 * 9-16) 
The ratio became 
TCARI 
OSAVI Ge) 


and is highly correlated with vegetation chlorophyll 
content (Haboudane et al., 2002). 


Visible Atmospherically Resistant Index—VARI 
Many resource managers would like vegetation frac- 
tion information (e.g., 60%) (Rundquist, 2002). Build- 
ing upon the Atmospherically Resistant Vegetation 
Index, scientists developed the Visible Atmospherically 
Resistant Index (VARI) computed as (Gitelson et al., 
2002): 


VARI _ P green ~ Pred 


(8.63) 
one Pereen * Pred~ Phiue 


The index is minimally sensitive to atmospheric effects, 
allowing estimation of vegetation fraction with an er- 


ror of <10% in a wide range of atmospheric optical 
thickness. 


Normalized Difference Built-Up Index—NDBI 
Many professionals working on urban/suburban prob- 
lems are interested in monitoring the spatial distribu- 
tion and growth of urban built-up areas. These data 
can be used for watershed runoff prediction and other 
planning applications. Zha et al. (2003) calculated a 
Normalized Difference Built-up Index (NDBD;: 


NDBI = B, — NDVI (8.64) 


where 
_ NIRpya— Mid IR pys 


a ; 
u NER pq + Mid IR pys 


(8.65) 


This resulted in an output image that contained only 
built-up and barren pixels having positive values while 
all other land cover had a value of 0 or —254. The tech- 
nique was reported to be 92% accurate. 


Vegetation Adjusted Nighttime Light (NTL) 
Urban Index—VANUI 

The Defense Meteorological Satellite Program/Opera- 
tional Linescan System (DMSP/OLS) Nighttime light 
(NTL) data are able to provide information on night- 
time luminosity which is highly correlated with the ex- 
istence of urbanized areas and energy consumption. 
Sometimes the utility of the NTL data is limited due to 


saturation of the data values, especially in downtown 
central business districts. Therefore, Zhang et al. (2013) 
developed the Vegetation Adjusted NTL Urban Index 
(VANUD, which incorporates MODIS-derived NDVI 
information along with NTL data: 


VANUI = (1—NDVI) x NTL. (8.66) 


The VANUI image exhibits reduced NTL saturation 
and corresponds to both biophysical (e.g., soil, water, 
vegetation) and urban characteristics. 


Red-Edge Position Determination—REP 

The abrupt change in the 680 — 800 nm region of re- 
flectance spectra of leaves caused by the combined ef- 
fects of strong chlorophyll absorption and leaf internal 
scattering is called the red edge. The red edge position 
(REP) is the point of maximum slope on a vegetation 
reflectance spectrum between the red and near-IR 
wavelengths. The red edge was first described by Col- 
lins (1978), and is perhaps the most studied feature on 
the vegetation spectral curve. The REP is useful be- 
cause it is strongly correlated with foliar chlorophyll 
content and can be a sensitive indicator of vegetation 
stress. Determining the red edge position using remote 
sensing data usually requires the collection of hyper- 
spectral data. 


A linear method proposed by Clevers (1994) can be im- 


plemented that makes use of four narrow bands and is 
computed as follows: 


P(red edge) P(700nm) 


REP = 700 + 40| ———=———- |,_ (8.67) 
P(740nm) — P(700nm) 
where 
_ P(670nm) * P(780nm) 
P(red edge) — 5) : (8.68) 


Dawson and Curran (1998) and Baranoski and Rokne 
(2005) summarize additional methods used to deter- 
mine the red edge position. 


Photochemical Reflectance Index—PRI 


Mand et al. (2010) used a photochemical reflectance 
index (PRI): 


_ (531 ~P570) 


PRI 
(P531 * P5709) 


(8.69) 


to monitor experimental warming and drought shrub 
land study areas in Europe. PRI was significantly relat- 
ed to effective green leaf-area-index (LAIe), however, it 
was found to be very sensitive to soil reflectance prop- 
erties especially in areas with low green LAIe. Sims et 
al. (2011) used PRI in their study of view angle effects 
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on MODIS data used in vegetation indices because it 
has been shown to be highly correlated with vegetation 
light-use-efficiency at leaf, canopy and stand levels. 


NDVI and Cellulose Absorption Index (CAI) to 
Quantify Fractional Cover of Photosynthetic 
Vegetation, Non-Photosynthetic Vegetation 

and Bare Soil 

Guerschman et al. (2009) state that quantitative esti- 
mation of fractional cover of photosynthetic vegeta- 
tion (fpy), non-photosynthetic vegetation which 
includes dead plant matter (fypy), and bare soil (fps) is 
critical for natural resource management and for mod- 
eling carbon dynamics. They developed a methodology 
based on the use of NDVI and a Cellulose Absorption 
Index (CAI) that can be used together in a linear un- 
mixing approach to identify the characteristics of the 
fractional cover. The CAI is computed using special 
short wavelength infrared (swir) regions of the electro- 
magnetic spectrum: 


CAI = [0.5 x (P79 + P27)—Po1] x 10 (8.70) 


where p>, = EO-1 Hyperion hyperspectral bands in 
the region from 2,022 — 2,032 nm; p >, = bands in the 
region from 2,093 — 2,113 and p 45 = bands in the re- 
gion from 2,184 — 2,204 (Guerschman et al., 2009). 
Fractional is estimated by determining where a pixel 
resides in the NDVI_CAT spectral feature space shown 
in Figure 8-49. The three endpoints in the feature space 
represent pure “endmembers” associated with bare soil 
(fgs), non-photosynthetic vegetation (fypy), and pho- 
tosynthetic vegetation (fpy). Reflectance spectra that 
are mixtures (e.g., at locations | to 4) of the three pure 
endmembers will lie within the yellow NDVI_CAI fea- 
ture space. In this hypothetical example, a pixel with 
#1 characteristics will likely consist of 50% bare soil 
and 50% photosynthetic vegetation. A pixel with #4 
characteristic will likely have equal proportions (e.g., 
33%) of each of the three endmembers. 


MERIS Terrestrial Chlorophyll Index (MTCI) 
Dash et al. (2010) used the MERIS Terrestrial Chloro- 
phyll Index (MTCI) to map the spatio-temporal varia- 
tion of vegetation phenology in India using the 
following equation: 


p —p p —p 
MTCI = band 10 band 9 _ 753.75 708.75 
Phand 9 Pband 8 708.757 681.25 


and is based on MERIS bands 8, 9, and 10. Their 
study used 8-day temporal composites of MERIS de- 
rived MTCI data. They suggest that the MTCI com- 
bines information on LAI and the chlorophyll 
concentration of leaves to produce an image of chloro- 
phyll content. The MTCI is an operational European 
Space Agency Level 2 land product. 


(8.71) 
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Conceptual Approach for Quantifying 
Vegetation Fractional Cover 
using Hyperspectral Data 
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FIGURE 8-49 Gureschman et al. (2009) suggest that it is 
possible to quantify fractional cover by analyzing informa- 
tion in the spectral feature space bounded by information 
from the Cellulose Absorption Index (CAI) and the Normal- 
ized Difference Vegetation Index (NDVI). Reflectance spec- 
tra that are mixtures (e.g., at locations 1-4) of the three pure 
endmembers will lie within the yellow feature space (based 
on Guerschman et al., 2009). 


Normalized Burn Ratio (NBR) 


One of the most widely used spectral indices for map- 
ping burn severity is the Normalized Burn Ratio 
(NBR) (Brewer et al., 2005). The NBR combines the 
reflectances in the near-infrared (p ,;.) and the short- 
wavelength infrared bands (p ,,,;,) (Boer et al., 2008): 


(Pir 7 Pewuie) 


NBR = 
(Pnir 1 Pcie) 


(8.72) 


The NBR is sometimes used to map fire-affected areas 
using a single post-fire image, but is more commonly 
applied to quantify burn severity as the difference (A ) 
between pre- and post-fire values observed in regis- 
tered multiple-date images (Boer et al., 2008): 


ANBR = NBR 


‘pre—fire — NBR 


post — fire (8.73) 
The NBR is used in the USDA Forest Service FIRE- 
MON system for the assessment and monitoring of fire 
impacts (Lutes et al., 2006; Boer et al., 2008). 


Vegetation Suppression 

Vegetation suppression algorithms model the amount 
of vegetation per pixel using a vegetation transform 
such as NDVI. The model calculates the relationship 
of each input band with vegetation, then it decorrelates 
the vegetative component of the total signal on a pixel- 


by-pixel basis for each band (Crippen and Blom, 2001; 
ENVI Suppression, 2013). Vegetation suppression is 
most commonly used in lithologic mapping and linear 
feature enhancement in areas with open canopies. This 
method helps analysts better interpret geologic and ur- 
ban features and works best with open-canopy vegeta- 
tion in medium spatial resolution (e.g., 30 <x 30 m) 
imagery. For closed canopies in moderate-resolution 
data, vegetation suppression is primarily used for lin- 
ear feature enhancement. 


Texture Transformations 


When humans visually interpret remotely sensed imag- 
ery, they synergistically take into account context, edg- 
es, texture, and tonal variation or color. Conversely, 
most digital image processing classification algorithms 
are based only on the use of the spectral (tonal) infor- 
mation (i.e., brightness values). Thus, it is not surpris- 
ing that there has been considerable activity in trying 
to incorporate some of these other characteristics into 
digital image classification procedures. 


A discrete tonal feature is a connected set of pixels that 
all have the same or almost the same gray shade 
(brightness value). When a small area of the image 
(e.g., a 3 X 3 area) has little variation of discrete tonal 
features, the dominant property of that area is a gray 
shade. Conversely, when a small area has a wide varia- 
tion of discrete tonal features, the dominant property 
of that area is texture. Most researchers trying to incor- 
porate texture into the classification process have at- 
tempted to create a new texture image that can then be 
used as another feature or band in the classification 
process. Thus, each new pixel of the texture image has 
a brightness value that represents the texture at that lo- 
cation (1.¢., BV; ; texture). 


First-Order Statistics in the Spatial 
Domain 


One class of picture properties that can be used for tex- 
ture synthesis is first-order statistics of local areas such 
as mean Euclidean distance, variance, skewness, kurto- 
sis (e.g., Luo and Mountrakis, 2010; Warner, 2011; 
Culbert et al., 2012; ERDAS, 2013). Typical algo- 
rithms include: 


Nie 
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Mean Euclidean Distance = 


(8.74) 


n—-1 


where 


BV... = brightness value of a window’s center pix- 
el for spectral band ~ 


BV,,. = brightness value for pixel (i,/) in spectral 
band 4 of a multispectral image 


n = number of pixels in the window. 


2 
VBV-w 
Variance = (8.75) 
n—-1 
where 
BV = brightness value of pixel (7,/) 
n = number of pixels in the window 
X = mean of the moving window where 
w= [ Br, /n. 
3 
» (BV;;—w) 
Skewness = 3 (8.76) 
4 
(n—1)(V) 
where V = Variance. 
4 
He Vs —p) 
Kurtosis = See KIES a (8.77) 
(n—1)(V) 


The pixel windows typically range from 3 x 3 to 5 x 5 
to 7 X 7. The application of a 3 x 3 Variance texture 
measure to the near-infrared band of a high spatial res- 
olution digital frame camera image is shown in Figure 
8-50b. The brighter the pixel, the greater the heteroge- 
neity (more coarse the texture) within the window. A 
composite of Variance derived from the NIR, red, and 
green bands is shown in Figure 8-50c. A color compos- 
ite of Skewness within the 3 x 3 spatial moving window 
derived from the NIR, red, and green bands is shown 
in Figure 8-50d. 


Numerous scientists have evaluated these and other 
texture transformations. For example, Mumby and Ed- 
wards (2002) extracted the variance from IKONOS im- 
agery using a 5 X 5 pixel window. They found that 
when texture features were used in conjunction with 
spectral data the thematic map accuracy increased for 
medium and fine levels of coral reef habitat. Ferro and 
Warner (2002) used a variance texture measure and 
found that land-cover class separability increased when 
texture was used in addition to spectral information, 
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and that texture separability increased with larger win- 
dows. Wang et al. (2004) used variance texture along 
with other second-order texture measures in their 
study of mangroves. Estes et al. (2008) used mean and 
standard deviation texture and Estes et al. (2010) used 
standard deviation texture in their analysis of forest 
antelope habitat using SPOT and ASTER data. Luo 
and Mountrakais (2010) used the mean and variance 
in their impervious surface study. Warner (2011) re- 
viewed the characteristics of numerous first-order tex- 
ture measures. Dronova et al. (2012) used standard 
deviation texture in their object-based image analysis 
of wetland plant function types. Gray and Song 
(2012) used simple variance texture in their studies of 
LAI extraction from MODIS data. 


Edge-Preserving Spectral-Smoothing (EPSS) 
Variance Texture 

Laba et al. (2010) created an iterative, edge-preserving 
spectral-smoothing (EPSS) texture measure associated 
with the mean and variance in a 5 X 5 spatial moving 
window (Figure 8-51) that was applied to individual 
bands of multispectral IKONOS imagery. The goal 
was to determine the mean of the central pixel, e.g., at 
location 23 in Figure 8-51a. The was performed by first 
computing the variance for a block of nine pixels cen- 
tered about a given pixel and for blocks of seven pixels 
that extend from the central pixel in eight directions 
about the central pixel as shown in Figure 8-51. The 
block with the minimum variance was then identified. 
Therefore, for each sub-window, k, the mean (Laba et 
al., 2010): 


3 3 
7 a os ae 


i=1j=1 


(8.78) 


and variance were computed 
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(8.79) 


where x;; was the brightness value at the i,j location 
within the Ath sub-window, and m;;; was a binary 
mask value (0 or 1) that corresponded with the Ath 
sub-window. The mean of the minimum variance sub- 
window was then assigned to the center pixel, x,, where 


X¢ =X, in Such that o ,i, = min (0 ,). (8.80) 


The method was applied to all the pixels in each band 
iteratively until a specified threshold was reached. 
Laba et al. (2010) extended the method to include in- 
formation on all the multispectral bands in the dataset. 
They found that this texture measure preserved abrupt 
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c. Color composite (RGB = NIR variance, 
red variance, green variance). 


b. Variance of the near-infrared (NIR) band. 


” { a i] 


. Color composite (RGB = NIR skewness, 
red skewness, green skewness). 


FIGURE 8-50 Selected first-order texture measures derived using a 3M 3 spatial moving window. a) Original image (original 
aerial photograph courtesy of John Copple, Sanborn Map Company). b) Variance of the near-infrared band. c) Composite of 
Variance derived from NIR, red, and green bands. d) Composite of Skewness derived from NIR, red, and green bands. 


spatial changes in spectral intensity or spectral shape 
found in the original data. 


Conditional Variance Detection 

Zhang (2001) developed a texture measure to identify 
tree features in high spatial resolution imagery. It 
consists of two components: 1) directional variance 
detection, and 2) standard local variance measure- 
ment, as previously discussed. The spatial logic of the 
directional variance detection is shown in Figure 8- 
52. Directional variance detection is used to identify 
whether the central pixel within the moving spatial 
window (e.g., 7 X 7) is located in a “treed” area. If the 
central pixel is in a treed area, the local variance cal- 
culation is then carried out to highlight the pixel in an 
output file. Otherwise, the local variance calculation 


is avoided to suppress the pixel. To effectively detect 
edges of other objects and separate them from treed 
areas, the size of the window for directional variance 
detection should be larger (e.g., 7 < 7) than the win- 
dow for local variance calculation (e.g., 3 X 3). 


The directional variance detector measures the pixel 
variances along the shaded pixels in Figure 8-52 on 
each side of the central pixel in four directions using 
the equation: 
1 n—-1 a) 
Dg = 7 » FaA-fGas] 


L==h 


(8.81) 


1 n-1 
FED =2 GA 


i=-n 


with 


Logic of the Edge-Preserving 
Spectral-Smoothing (EPSS) 
Texture Measure 


xX, = pixel to be Wled 
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Nine Different Groups (Sub-windows) of Pixels 


| [34] 35 


rae os j.3X3 SE. 


FIGURE 8-51 The logic of the Edge-Preserving Spectral-Smoothing (EPSS) texture measure. a) The 5X 5 pixel spatial moving 
window with the pixel to be filled identified. b-j) Nine different groups of pixels are evaluated to come up with the mean of the 
minimum-variance sub-window which is then assigned to x,. Eight of the groups are associated with the cardinal directions. 
The method can be extended to work with multispectral data (based on Laba et al., 2010). 


and 
i<0,7=0 for the upper side 
i> 0,7 = 0 for the lower side 
i= 0,7 <0 for the left side 
i> 0,7 > 0 for the right side 


where D,,,, is directional variance, f (i,j) is the value of 
the pixel located at the ith row and jth column in the 
spatial moving window (Figure 8-52), and 7 is the pix- 
el count on each side of the central pixel. 


If the variance on one of the four sides is less than a 
user-specified threshold, it is concluded that there is a 
homogenous area on this side or the central line is 
along a straight edge. In this case, the central pixel is 
regarded as a non-tree pixel. It is assigned a lower val- 
ue and the local variance calculation is not carried out. 
If, on the other hand, the variance in one of the direc- 
tions exceeds the threshold, then the calculation of the 
local variance (e.g., within a 3 x 3 window) is carried 
out and the central pixel receives a higher value. Zhang 
(2001) determined that the algorithm detected tree fea- 
tures more accurately than gray-level co-occurrence 
matrix-derived texture algorithms, to be discussed. 


Window Used to Compute 
Directional Variance 
3 2 -1 O 1 2 3 


Central pixel 


Pixels on 
four sides 


FIGURE 8-52 Window used to compute directional vari- 
ance (based on Zhang, 2001). It does not have to be a 
7 & 7 window. 


Min-—Max Texture Operator 

Briggs and Nellis (1991) developed a min-max texture 
operator based on the analysis of the brightness values 
found within the following five-element spatial moving 
window: 
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Moran’s J Calculation 


Z, = pixel to be Wled 


1 


79 |7 [32[48] 2x2 | 
ve pA Liee2|zo[s0) ones ae” 
“ 7m at | 10 | 17 [st [16 Z = (50+43+45+50)/4 = 47 
1 sj Ly 


a. Original brightness values 
in a band of imagery. 
W,, (Adjacency matrix) 
att Z2 23 ZA 


Moran’s I 


C,,= (Z,- Z(Z-Z) 
pes Z2 Z3 ZS 


S’=X(Z,- Zin 
S°= (9+ 16+4+9)/4 = 9.5 
L2C,, i= _ (0 x 9)+1 x— -12)+(1 x —6)+... = 27) 


= 0.947 


f. Moran’s J value 
in the output matrix. 


FIGURE 8-53 Hypothetical example of computing Moran’s | to derive texture in a single band of imagery (based on Purkis et 


al., 2006). 


where 
Texturea = brightest 4p C.D, E~ darkest 4 B,C,DE (8.82) 


They found that the min-max texture features and 
NDVI transformations of seven SPOT HRV scenes 
provided accurate information on the seasonal varia- 
tion and heterogeneity of a portion of the tallgrass 
Konza Prairie Research Natural Area in Kansas. 


Moran’s | Spatial Autocorrelation as a Texture 
Measure 

Purkis et al. (2006) used the Moran’s J spatial autocor- 
relation metric as a texture operator in their coral reef 
mapping research. Moran’s J was calculated using 
(Purkis et al., 2006): 


uae 
Se a ee (8.83) 


ILA, 
ij 


where W; ;is the weight at distance d so that W;; = 1 if 
point 7 is within distance d from point i; otherwise, Wes 
= 0; Cy are deviations from the mean (ie. 

= = (Z,- Z)(Z;— Z) where Z is the mean bright- 
ness ees in the local window. S’ is the mean of the 
major diagonal of C;. Moran’s J is computed within a 


user-specified moving window (e.g., 2 X 2, 3 x 3). An 


Morans I 
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Hypothetical Example of Creating a Gray-level Co-occurence Matrix (GLCM) 
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FIGURE 8-54 Hypothetical example of computing the elements using a 5% 5 kernel applied to single band of imagery. a) 
Original brightness values in the 5X 5 area. b) Number of times brightness value i occurs to the left of brightness value j. c) 
Number of times brightness value i occurs to the right of brightness value j. d) Symmetric. e) Normalization of the matrix. f) 


GLCM matrix ready for use with algorithms. 


example of how Moran’s J is calculated using hypo- 
thetical data in a simple 2 x 2 spatial moving window is 
shown in Figure 8-53 (Purkis et al., 2006). Moran’s J 
varies from +1.0 for positive correlation (a clumped 
pattern) to —1.0 for negative correlation. If positive au- 
tocorrelation exists, pixels with similar spectral charac- 
teristics can be inferred to be present within the 
window, 1.e., homogeneous texture is present. If spatial 
autocorrelation is weak or nonexistent, adjacent pixels 
within the window have dissimilar spectral values, Le., 
heterogenous texture is present (Purkis et al., 2006). 


Second-Order Statistics in the Spatial 
Domain 


A suite of very useful texture measures was originally 
developed by Haralick and associates (Haralick et al., 
1973; Haralick and Shanmugan, 1974; Haralick, 1979; 
Haralick and Fu, 1983; Haralick, 1986). The higher- 
order set of texture measures is based on brightness 
value spatial-dependency gray-level co-occurrence ma- 
trices (GLCM). The GLCM-derived texture transfor- 
mations continue to be very widely adopted by the 
remote sensing community and are often used as an 
additional feature in multispectral classification (e.g.; 
Maillard, 2003; Hall-Beyer, 2007; Schowengerdt, 2007; 
Jensen et al., 2009; Luo and Mountrakis, 2010; Pena- 
Barragan et al., 2011; Warner, 2011; Culbert et al., 
2012; Wang and Zhang, 2014). 


But how are these higher-order texture measures com- 
puted? If c = (Ax ,Ay) is considered to be a vector in 


the (x, y) image plane, for any such vector and for any 
image f (x, y) it is possible to compute the joint proba- 
bility density of the pairs of brightness values that oc- 
cur at pairs of points separated by c. If the brightness 
values in the image can take upon themselves any val- 
ue from 0 to the highest quantization level in the image 
(e.g., quant; = 255), this joint density takes the form of 
an array h,., where /,(i, j) is the probability that the 
pairs of brightness values (i, /) occur at separation c. 
This array /, is quant, by quant, in size. It is easy to 
compute the h, array for f(x, v), where Ax and Ay are 
integers by simply counting the number of times each 
pair of brightness values occurs at separation c = (Ax 
and Ay ) in the image. For example, consider the sim- 
ple image that has just five lines and five columns and 
contains brightness values ranging from only 0 to 3 in 
Figure 8-54a. If (Ax and Ay) = (1, 0), then the num- 
bers are represented by the brightness value spatial-de- 
pendency matrix /jy o¢ Shown in Figure 8-54b where 
the entry in row i a column j of the matrix is the 
number of times brightness value i occurs to the left of 
brightness value j. For example, brightness value 1 is 
to the left of brightness value 2 a total of three times 
(i.e., A,[1, 2] = 3 in the GLCM). 


However, in order to produce a “symmetric” GLCM, 
it is good practice to create a GLCM that also tracks 
the number of times value i occurs to the right of 
brightness value j as shown in Figure 8-54c. These two 
GLCMs are then added together to create the symmet- 
ric GLCM (Hall-Beyer, 2007) shown in Figure 8-54d. 
The GLCM is then normalized by dividing every cell 
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in it by the total number of possible values. In this case 
every cell in the matrix is divided by 40. The normal- 
ized, symmetrical GLCM (/,) is now ready to be used 
in the various texture measures. 


It is assumed that all textural information is contained 
in the brightness value spatial-dependency matrices 
that are developed for angles of 0X 45 90M and 135X 
(Figure 8-55). Generally, the greater the number of val- 
ues found in the diagonal of the co-occurrence matri- 
ces, the more homogeneous the texture is for that part 
of the image being analyzed. 


There are a variety of measures that can be used to ex- 
tract useful textural information from the /, matrices. 
Some weight the off-diagonal elements in the /, matri- 
ces more heavily than the diagonal elements (e.g., Hall- 
Beyer, 2007). Several of the more widely used GLCM 
texture measures are the angular second moment 
(ASM), contrast (CON), correlation (COR), dissimi- 
larity (DIS), entropy (ENT>) and homogeneity (HOM) 
(Haralick, 1986; Gong et al., 1992; Zhang, 2001; Mail- 
lard, 2003; Kayitakire et al., 2006; Luo and Mountra- 
kis, 2010; Warner, 2011; Wood et al., 2012): 


quant, quant, 


ASM = 2 X A Als ay (8.84) 
=0 j=0 
quant, quant, . ; 
CON= YOO Dy G-s x he) (8.85) 
i=0 j=0 
quant, quant, ij- REG 2 
COR = Fy LMA agg 
i=0 j=0 © 
quant, quant, 
DIS = »y x hj li-Jl (8.87) 
i=0 j=0 
quant, quant, 
ENT, = > x hij) x log[h Gj] (8.88) 
i=0 j=0 
quant, quant, 
HOM = » ee 5A CGF) (8.89) 


=e 


where 


quant, = quantization level of band k (e.g., 2° = 010255) 
hG, j) = the (i j)th entry in one of the angular brightness 
value spatial-dependency matrices, 


135° a 45° 


w—- 


FIGURE 8-55 The eight nearest neighbors of pixel X ac- 
cording to angle ¢ used in the creation of spatial depen- 
dency matrices for the measurement of image texture. 


and 


quant, quant, 


2 DY) xh GA) 


=0 j=0 


MEAN (1) = (8.90) 


quant, quant, 


VARIANCE (07) = > » Gi ee SD) 
i=0 j=0 


Note that the MEAN (un) and VARIANCE (07), 
which are also texture measures, use the elements of 
the GLCM rather than the original brightness values 
in the image during computation (Hall-Beyer, 2007). 
Four brightness value spatial-dependency matrices (0X 
45x, 90M and 135X) can be derived for each pixel based 
on neighboring pixel values. The average of these four 
measures can be output as the texture value for the pix- 
el under consideration. Most analysts, however, use 
just one direction (e.g., horizontal or vertical) to create 
the GLCMs used with the various algorithms. 


To create a GLCM-derived texture measure, the ana- 
lyst usually has to make several important decisions, 
including (Warner, 2011): 


the texture measure(s), 

window size (e.g., 3 X 3, 5 X 5, 32 x 32) 

input channel (i.e, the spectral channel used to 
extract texture information) 


quantization level of the input data (e.g., 8-bit, 6-bit, 
4-bit) used to produce the output texture image, and 


¢ the spatial component (i.e., the interpixel distance 
and angle used during co-occurrence computation). 


It is possible to use any GLCM-derived texture mea- 
sure alone. However, Clausi (2002) found that there 
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Second-order Texture Information Extracted from Digital Frame Camera Imagery 


a. Color composite (RGB = NIR, red, green). 


“ne 


omy 


d. Composite (RGB = NIR Dissimilarity, 
red Dissimilarity, green Dissimilarity). 


g. Composite (RGB = NIR Homogeneity, 
red Dissimilarity, green Contrast). 


FIGL 


b. Composite (RGB = NIR Angular 
Second Moment, red ASM, green ASM). 
. oe ae 


e. Composite (RGB = NIR Entropy, 
red Entropy, green Entropy). 


eee ee od cal 


h. Composite (RGB = NIR Mean, 
red Homogeneity, green Dissimilarity). 


c. Composite (RGB = NIR Contrast, 
red Contrast, green Contrast). 


f. Composite (RGB = NIR Homogeneity, 
red Homogeneity, green Homogeneity). 


i. Composite (RGB = NIR Mean, 
red Homogeneity, green Dissimilarity). 


56 Selected texture measures derived from gray-level co-occurrence matrices (GLCM). a) Original image. b) Angu- 


lar Second Moment. c) Contrast. d) Dissimilarity. e) Entropy. f) Homogeneity. g) Composite of Homogeneity, Dissimilarity, and 
Contrast. h) Composite of Mean, Homogeneity, and Dissimilarity. i) Composite of Mean, Contrast and Dissimilarity (original 
aerial photography courtesy of John Copple, Sanborn Map Company). 


was a preferred subset of statistics (contrast, correla- 
tion, and entropy) that was superior to the use of any 
single statistic or using the entire set of statistics. Gong 
et al. (1992) found that 3 x 3 and 5 x 5 windows were 
generally superior to larger windows. Conversely, Ferro 
and Warner (2002) found that land-cover texture sepa- 
rability increased with larger windows. Several scien- 
tists suggest that it is a good idea to reduce the 
quantization level of the input data (e.g., from 8-bit 
data with values from 0 to 255 to 5-bit data with values 


from 0 to 31) when creating the texture images so that 
the spatial-dependency matrices to be computed for 
each pixel do not become too large (Clausi, 2002). 


Examples of several GLCM-based texture measures 
are shown in Figure 8-56b-f. The texture images shown 
were derived for each of the three bands in the dataset 
(green, red, and NIR). It is clear from these examples 
that the individual texture measures yield different tex- 
ture information. Additional insight is gained when 
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color-composites are based on the use of separate tex- 
ture measures from the three bands, e.g., homogeneity, 
dissimilarity, and contrast in Figure 8-56g, and mean, 
homogeneity, and dissimilarity in Figure 8-56h, and 
mean, contrast, and homogeneity in Figure 8-561. 


Herold et al. (2003) investigated a variety of land- 
scape ecology spatial metrics and GLCM texture 
measures to classify urban land use in IKONOS high 
spatial resolution imagery. Wang et al. (2004) used the 
G_CM texture measures recommended by Cau si 
(2002) in their study of mangroves. Tuominen and 
Peekarinen (2005) used five GLCM texture measures 
applied to aerial photography to inventory forests. 
Frohn and Hao (2006) used GLCM-derived texture 
in their evaluation of landscape metric performance 
using Landsat data of Rondonia, Brazil. Kayitakire et 
al. (2006) used GLCM Variance, CON, and COR tex- 
ture measures with four directions and three window 
sizes (5 X 5, 15 X 15, and 25 x 25) in their study of for- 
est texture using high spatial resolution IKONOS-2 
imagery. Hall-Beyer (2007) provides a very useful tuto- 
rial on how to compute GCLM-based texture mea- 
sures and discusses some of the important decisions 
that must be made when using them. Estes et al. (2008) 
used the ASM in their analysis of forest antelope habi- 
tat using SPOT and ASTER data. Culbert et al. (2012) 
had good success using GLCM texture measures ap- 
plied to 114 Landsat scenes when modeling broad- 
scale patterns of avian species richness across the 
Midwestern U.S. Luo and Mountrakais (2010) used 
CON, Energy, and HOM in their impervious surface 
study. Warner (2011) summarized the characteristics of 
many GLCM texture measures and suggested ways to 
overcome challenges of selecting the texture metric, 
kernel size, spectral band, radiometric quantization, 
displacement, and angle. Pena-Barragan et al. (2011) 
used HOM, and ENT in their object-based crop identi- 
fication. Dronovia et al. (2012) used ENT and HOM 
texture metrics in their object-based image analysis of 
wetland plant function types. Wood et al. (2012) used 
eight GLCM texture measures in their study of grass- 
land-savannah-woodland vegetation structure using 
aerial photographs and Landsat TM data. Zhang and 
Xie (2012) investigated the use of GLCM-based tex- 
ture in their object-based analysis of hyperspectral 
data of the Florida Everglades. 


Texture Units as Elements of a 
Texture Spectrum 


Wang and He (1990) computed texture based on an 
analysis of the eight possible clockwise ways of order- 
ing the 3 x 3 matrix of pixel values shown in Figure 8- 
57a. This represents a set containing nine elements V = 
{Vo, Vi, -.-. Vg}, with Vp representing the brightness 
value of the central pixel and V; the intensity of the 


neighboring pixel 7. The corresponding texture unit is a 
set containing eight elements, TU = {£), Ep, ..., Es}, 
where E; is computed: 
fori=1,2,...,8 

E,= Oi V;<¥o 
E,=1if Vi=V, (8.92) 


E, 
i 


2 if V,>V% 


and the element £; occupies the same position as pixel 
i. Because each element of TU has one of three possi- 
ble values, the combination of all eight elements results 
in 3° = 6,561 possible texture units. There is no unique 
way to label and order the 6,561 texture units. There- 
fore, the texture unit of a 3 x 3 neighborhood of pixels 
(Figure 8-57b—d) is computed: 


(8.93) 


where £; is the ith element of the texture unit set TU = 
{E|, Eo, ..., Eg}. The first element, E;, may take any 
one of the eight possible positions from a through / in 
Figure 8-57a. An example of transforming a 3 x 3 
neighborhood of image brightness values into a texture 
unit (TU) and a texture unit number (Vy) using the 
ordering method starting at a is shown in Figure 8-57. 
In this example, the texture unit number, Nyy, for the 
central pixel has a value of 6,095. The eight brightness 
values in the hypothetical neighborhood are very di- 
verse (that is, there is a lot of heterogeneity in this small 
region of the image); therefore, it is not surprising that 
the central pixel has such a high texture unit number. 
Eight separate texture unit numbers could be calculat- 
ed for this central pixel based on the eight ways of or- 
dering shown in Figure 8-57a. The eight Nyy values 
could then be averaged to obtain a mean Ny value for 
the central pixel. 


The possible texture unit values range from 0 to 6,560 
and describe the local texture of a pixel in relationship 
to its eight neighbors. The frequency of occurrence of 
all the pixel texture unit numbers over a whole image is 
called the texture spectrum. It may be viewed in a graph 
with the range of possible texture unit numbers (Vy) 
on the x-axis (values from 0 to 6,560) and the frequen- 
cy of occurrence on the y-axis (Figure 8-58). Each im- 
age (or subimage) should have a unique texture 
spectrum if its texture is truly different from other im- 
ages (or subimages). 


Wang and He (1990) developed algorithms for extract- 
ing textural features from the texture spectrum of an 


Possible 
clockwise 
ways of 


ordering Neighborhood 


V = {40, 63, 28, 45, 35, 
21, 40, 67, 88} 


a. b. 
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Texture Unit 


Texture Unit Number 
Nyy = 6095 
TU ={2, 0, 2, 0, 0, 1, 2, 2} 
iC: d. 


FIGURE 8-57 How a3 3 neighborhood of brightness values is transformed into a texture unit number (Nyy), which has val- 
ues ranging from 0 to 6,560. a) Possible clockwise ways of ordering the eight elements of the texture unit. The first element E; 
in Equation 8.93 may take any of the eight positions from a through h. In this example, the ordering position begins at a. b) 
Brightness values found in the 3 3 neighborhood. This is a very heterogeneous group of pixels and should result in a high 
texture unit number. c) Transformation of the neighborhood brightness values into a texture unit. d) Computation of the tex- 
ture unit number based on Equation 8.93 (values range from 0 to 6,560). It is possible to compute eight separate texture unit 
numbers from this neighborhood and then take the mean (based on Wang and He, 1990). 


image, including black-white symmetry, geometric 
symmetry, and degree of direction. Only the geometric 
symmetry measure is presented here. For a given tex- 
ture spectrum, let S(i) be the occurrence frequency of 
the texture unit numbered 7 in the texture spectrum un- 
der the ordering way j, where i = 0, 1, 2, ..., 6560 and j 
= 1, 2, 3, ..., 8 (the ordering ways a, b, c ..., h are, re- 
spectively, represented by 7 = 1, 2, 3, ..., 8). Geometric 
symmetry (GS) for a given image (or subimage) is: 


6560 
ee ee 
GS = a x 100. (8.94) 
eT eo) 
-=6 


GS values are normalized from 0 to 100 and measure 
the symmetry between the spectra under the ordering 
ways a and e, b and f, c and g, and d and h for a given 
image. This measure provides information on the shape 
regularity of images (Wang and He, 1990). A high val- 
ue means the texture spectrum will remain approxi- 
mately the same even if the image is rotated 180 The 
degree of direction measure provides information 
about the orientation characteristics of images. 


Texture Statistics Based on the 
Semi-variogram 

Numerous authors have investigated the use of the 
semi-variogram discussed in Chapter 4 to derive tex- 
ture information (e.g., Woodcock et al., 1988; Lark, 
1996; Rodriguez-Galiano et al., 2012). Maillard (2003) 
identified the problems associated with using different 


Texture Spectrum 


Frequency of 
Occurence 


0 6,560 


Texture Unit Number (N,,,,) 
FIGURE 8-58 Hypothetical texture spectrum derived from 
an image or subimage. 


variogram models (e.g., spherical, exponential, and si- 
nusoidal) and other criteria. He developed a variogram 
texture operator that: 


* uses a rather large window to cover larger distance 
lags (up to 32 pixels), 
* is rotation-invariant and preserves anisotrophy, and 


* incorporates the mean square-root pair difference 
function (SRPD) as a semi-variance estimator. 


Maillard compared the variogram, GLCM, and Fouri- 
er-based texture measures and concluded that the var- 
iogram and GLCM texture measures were generally 
superior. Rodriguez-Galiano et al. (2012) used five dif- 
ferent types of geostatistical textural measures for 
three different window sizes and three different lags 
yielding a total of 972 potential input variables. They 
found the geostatistical texture measures to be more 
useful than the GLCM texture measures. 
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Although texture features have been increasingly in- 
corporated into multispectral classifications, no single 
algorithm that combines efficiency and effectiveness 
has yet to be widely adopted. Also, the texture features 
derived for one type of application (e.g., land-use clas- 
sification at the urban fringe) are not necessarily useful 
when applied to another geographic problem (e.g., 
identification of selected geomorphic classes). Finally, 
some parameters central to the computation of the tex- 
ture features are still derived empirically (e.g., the size 
of the window or the location of certain thresholds). 
This makes it difficult to compare and contrast studies 
when so many variables in the creation of the texture 
features are not held constant. 


Landscape Ecology Metrics 


Vegetation indices are useful for monitoring the condi- 
tion and health of vegetated pixels. However, such per- 
pixel analysis does not provide any information about 
the nature of surrounding pixels, including their site 
and association characteristics. Landscape ecology 
principles have been developed that increasingly incor- 
porate remote sensor data to assess the health and di- 
versity of vegetation and other variables within entire 
ecosystems. This has resulted in the development of 
numerous landscape ecology metrics or indicators that 
are of significant value when analyzing rangeland, 
grassland, forests, and wetland (Frohn, 1998; Frohn 
and Hao, 2006; McGarigal et al., 2013). Numerous 
government agencies, such as the Environmental Pro- 
tection Agency, base much of their environmental 
modeling and landscape characterization on these met- 
rics and indicators (EPA Landscape Ecology, 2014). 
Therefore, it is useful to provide a brief review of their 
origin and summarize several of the more important 
landscape ecology metric variables that can be extract- 
ed from remotely sensed data. 


The term landscape ecology was first introduced by the 
German geographer Carl Troll (1939) who made wide- 
spread use of the then new technique of aerial photog- 
raphy. Troll intended for the term landscape ecology to 
distinguish his approach for using such imagery to in- 
terpret the interaction of water, land surfaces, soil, veg- 
etation, and land use from that of conventional 
photographic interpretation and cartography. Land- 
scape ecology has been intensively practiced in Europe 
for many decades and became generally recognized in 
the United States in about 1980. Since then, landscape 
ecology has rapidly evolved as a discipline, spurred by 
the synergistic interactions between remote sensing 
and GIS techniques and advances in ecological theory. 


Landscape ecology is the study of the structure, func- 
tion, and changes in heterogeneous land areas com- 
posed of interacting organisms (Bourgeron and Jensen, 
1993). It is the study of the interaction between land- 
scape patterns and ecological processes, especially the 
influence of landscape pattern on the flows of water, 
energy, nutrients, and biota. What distinguishes land- 
scape ecology from the many separate disciplines that 
it embraces (e.g., geography, biology, ecology, hydrolo- 
gy) is that it provides a hierarchical framework for in- 
terpreting ecological structure, function, change, and 
resiliency at multiple scales of inquiry. 


Traditional measures to protect the environment, such 
as preventing water pollution or protecting biodiver- 
sity, often focused on specific effluent discharges or 
fine-scale habitat requirements. This method has been 
described as the fine-filter approach. In contrast, the 
coarse-filter approach to resource conservation states 
that “by managing aggregates (e.g., communities, eco- 
systems, landscapes), the components of these aggre- 
gates will be managed as well” (Bourgeron and Jensen, 
1993). In other words, the most cost-effective strategy 
to maintain the resiliency and productivity of ecologi- 
cal systems is to conserve (or restore) the diversity of 
species, ecosystem processes, and landscape patterns 
that create the systems. Applying this coarse-filter 
management method requires that landscape patterns 
be evaluated at multiple spatial and temporal scales 
rather than simply at the traditional scales of stream 
reach or forest stand. 


Hierarchy theory allows landscape ecologists to inte- 
grate multiple scales of information to determine 
whether landscape patterns are sufficient to allow eco- 
logical processes to operate at the necessary scales. The 
objective is to investigate changes in the distribution, 
dominance, and connectivity of ecosystem components 
and the effect of these changes on ecological and bio- 
logical resources. For example, ecosystem fragmenta- 
tion has been implicated in the decline of biological 
diversity and ecosystem sustainability at a number of 
spatial scales. Determining status and trends in the pat- 
tern of landscapes is critical to understanding the over- 
all condition of ecological resources. Landscape 
patterns thus provide a set of indicators (e.g., pattern 
shape, dominance, connectivity, configuration) that 
can be used to assess ecological status and trends at a 
variety of scales. 


A hierarchical framework also permits two important 
types of comparisons: 1) to compare conditions within 
and across landscapes, and 2) to compare conditions 
across different types of ecological risks. Such ecologi- 
cal risks include the risk of erosion, loss of soil produc- 
tivity, loss of hydrologic function, and loss of 
biodiversity. 
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TABLE 8-11 Societal values, example indicators, and candidate landscape ecology metrics. 


Societal Value 


Indicator 


Candidate Metrics 


Biodiversity 


wildlife habitat suitability 


patch statistics (number, total area, average size, largest size, distance 
between, ratio perimeter to area, shape, fractal dimension, square 
pixel model, etc.), fragmentation, contagion, zone fragmentation 
index, patch per-unit-area index, dominance, adjacency of land-cover 
types, Shannon diversity, biophysical attribute patterns 


stream biological condition 


diversity, square pixel model, dominance, fragmentation, zone frag- 
mentation index, patch per-unit-area index, adjacency of land-cover 
types, slope, elevation, diffusion rates, percolation threshold, erosion 
index, texture, biophysical attribute patterns, geochemical attributes 


forest plant species richness 


diversity, dominance, fragmentation, zone fragmentation index, patch 
per-unit-area index, slope, erosion index, texture, patch statistics, 
square pixel model, biophysical attribute patterns 


landscape sustainability 


patch statistics, contagion, zone fragmentation index, patch per-unit- 
area index, fragmentation, texture, dominance, fractal dimension, 
square pixel model, biophysical attribute patterns 


Watershed integrity 


water quality 


patch statistics, erosion index, hydrologic modification, adjacency of 
land-cover types, dominance, contagion, zone fragmentation index, 
patch per unit area index, fractal dimension, square pixel model, ele- 
vation, slope, biophysical attribute patterns, geochemical attributes 


vulnerability to flooding 


patch statistics, adjacency of land-cover types, erosion index, domi- 
nance, contagion, zone fragmentation index, patch per-unit-area 
index, fractal dimension, square pixel model, hydrologic modification, 
elevation, slope, texture, biophysical attribute patterns 


Landscape resilience 


landscape sustainability 


patch statistics, dominance, contagion, zone fragmentation index, 
patch per-unit-area index, fragmentation, fractal dimension, square 


pixel model, biophysical attribute patterns 


Scalable units are needed to address landscape ecology 
issues at multiple scales within a hierarchical frame- 
work. Examples of scalable units include patches, pat- 
terns, and landscapes. A patch unit is a set of 
contiguous measurement units (e.g., pixels) that have 
the same numerical value. A pattern unit is a collection 
of measurement units and/or patch units that have the 
property of being the minimum unit descriptor of a 
larger spatial area. The scales of assessment questions 
and indicators suggest two types of landscape units: 
watersheds and landscape pattern types (LPT) (Wick- 
ham and Norton, 1994). Watersheds and LPTs capture 
or bound four important flow processes operating 
within and among landscapes: flows of energy, water, 
nutrients, and biota. Scales of watersheds and LPTs 
range from approximately 10° to 10° units in extent, 
and from | to 100 ha. 


Landscape Indicators and Patch Metrics 


Jones et al. (1998) suggest that landscape integrity can 
be monitored by carefully watching the status of the 
following indicators: 


¢ land-cover composition and pattern 
* riparian extent and distribution 


* ground water 

* greenness pattern 

¢ degree of biophysical constraints 
* erosion potential. 


Monitoring these landscape indicators requires precise, 
repeatable measurements of terrain patches such as in- 
dividual forest stands, rangeland, wetland, and/or agri- 
cultural fields (Table 8-11). It is also important to 
identify patches of pure urban structure such as resi- 
dential and commercial land use. These measurements 
of terrain patches are routinely referred to as /andscape 
pattern and structure metrics (McGarigal et al., 2013). 
Numerous landscape structure metrics have been de- 
veloped (e.g., Weiers et al., 2004; Frohn and Hao, 
2006). Many of these landscape ecology metrics are 
available in FRAGSTATS—A Spatial Pattern Analysis 
Program for Categorical Maps (McGarigal et al., 
2013). Scientists often use this program to extract 
patch, class, and landscape indices. 


O’Neill et al. (1997) suggest that the health of an eco- 
system could be monitored if the following three land- 
scape ecology metrics were monitored through time: 
dominance, contagion, and fractal dimension. 
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Dominance, D, is the information theoretic index that 
identifies the extent to which the landscape is dominat- 
ed by a single land-cover type. The metric, 0 < D < 1, is 
computed as 


p11 


where 0 < P; <1 is the proportion of land-cover type k, 
and n is the total number of land-cover types present in 
the landscape. 


(Ps aa (8.95) 


In(n) 


Contagion, C, expresses the probability that land cover 
is more “clumped” than the random expectation. The 
index, 0 < C <1, is 


aia es 


where P;; is the an that a pixel of cover type / is 
adliacent to type j. 


ii * if 
| (8.96) 
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The fractal dimension, F, of patches indicates the extent 
of human reshaping of landscape structure (O’Neill et 
al., 1997). Humans create simple landscape patterns; 
nature creates complex patterns. The fractal dimension 
index is calculated by regressing the log of the patch 
perimeter against the log of the patch area for each 
patch on the landscape. The index equals twice the 
slope of the regression line. Patches of four or fewer 
pixels are excluded because resolution problems distort 
their true shape. 


O’Neill et al. (1997) suggest that this set of three indi- 
ces may capture fundamental aspects of landscape pat- 
tern that influence ecological processes. Significant 
changes in these indices for an ecosystem might indi- 
cate that perhaps deleterious processes are at work in 
the environment. For example, consider a small ecosys- 
tem that exhibits a less modified fractal dimension, 1s 
highly clumped, and has relatively few land-cover types 
within it. It might appear in a three-dimensional land- 
scape metric space at location a in Figure 8-59. If this 
small ecosystem were subdivided with several new 
roads and fragmented, its location might move in 
three-space to b with many land-cover types being in- 
troduced (dominance change), be less clumped (conta- 
gion), with a more modified fractal dimension. This 
could be good or bad. In fact, the relationships be- 
tween the metric values and how they actually relate to 
ecological principles are still being determined. Re- 
mote sensing of vegetation within these patches is very 
important and constitutes one of the major factors re- 
sponsible for whether or not the metrics are robust and 
useful for ecological modeling. 


Landscape Metric Space 


less 
modified 


many land 
cover types 


Fractal Dimension, F 


al 
< 
oe 
y 
few land 
cover types 


more 


modified Contagion, C 


clumped not clumped 


FIGURE 8-59 A hypothetical three-dimensional landscape 
metric feature space. Point a represents where a stable, un- 
modified ecosystem might reside in dominance, contagion, 
and fractal dimension feature space. When the landscape is 
modified by man or natural forces, it may move to point b 
with many land-cover types being introduced (dominance 
change), be less clumped (contagion), and exhibit a more 
modified fractal dimension. This could be good or bad. Re- 
mote sensing of vegetation is critical to the extraction of 
landscape patch metrics. 


Ritters et al. (1995) reviewed 55 patch metrics and con- 
cluded that the following metrics accounted for most 
of the variance, including: the number of attribute cov- 
er types in a region (n); contagion (previously dis- 
cussed); average perimeter-area ratio; perimeter-area 
scaling; average large-patch density-area scaling; stan- 
dardized patch shape; and patch perimeter-area scal- 
ing. The algorithms for all 55 metrics are provided in 
the paper. 


Applying the principles of landscape ecology requires 
an understanding of the natural variability of land- 
scape patterns and processes across both space and 
time. Estimates of this variability are essential to deter- 
mining whether the current condition of landscape is 
sustainable, given its historic patterns and processes. 
Moreover, the estimates are extremely useful in both 
broad-level assessment of risk to resources, as well as 
to finer-scale assessments. 


The goal is usually to 


* estimate, on a regional basis and with known confi- 
dence, the current status, trends, and changes in 
selected indicators of the important landscapes; 


* estimate with known confidence the geographic cov- 
erage and extent of the landscapes’ patterns and 


types; 


* seek associations between selected indicators of nat- 
ural and anthropogenic stressors and indicators of 
landscape condition; and 


* provide statistical summaries and periodic assess- 
ments of the condition of the landscapes. 


Monitoring the vegetation in landscape patches using 
remotely sensed data and deriving accurate metrics 
from these data is a very important component of 
many ecosystem monitoring programs. For example, 
Arroyo-Moya et al. (2005) used remote sensing-derived 
landscape ecology metrics to analyze forest fragmenta- 
tion and regrowth characteristics in Costa Rica from 
1960-2000. Jackson and Jensen (2005) used remote 
sensing-derived landscape ecology metrics to perform 
reservoir shoreline environmental monitoring in South 
Carolina. Frohn and Hao (2006) provide an overview 
of numerous landscape ecology metrics applied to their 
study of deforestation in Rondonia, Brazil. 
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9 THEMATIC INFORMATION EXTRACTION: 
PATTERN RECOGNITION 


Remote sensing data are collected by an increasingly 
diverse array of airborne and satellite remote sensing 
systems (e.g., Colomina and Molina, 2014; Belward 
and Skoien, 2014). The data can be processed and 
transformed into various types of thematic informa- 
tion. In particular, remote sensing has increasingly 
been used as a source of land-use and land-cover infor- 
mation at local, regional, and global scales (Homer et 
al., 2012; Jensen and Jensen, 2013). The extraction of 
thematic information is often performed by identifying 
spectral and/or spatial patterns in the remote sensor 
data that are correlated with different types of land-use 
or land-cover present. The term pattern recognition is 
commonly used to describe this analytical process. 


Ne Overview 


This chapter reviews the fundamental methods used to 
apply pattern recognition techniques to multispectral 
data. It begins by reviewing the general steps required 
to extract land-cover information from digital remote 
sensor data. The concept of hard (crisp) versus soft 
(fuzzy) classification logic is introduced. The charac- 
teristics of several of the most important land-cover 
classification schemes are reviewed. Numerous meth- 
ods of supervised classification (e.g., parallelepiped, 
minimum distance, maximum likelihood) and unsuper- 
vised clustering (e.g., ISODATA) are discussed. Fea- 
ture selection methods used to identify the most 
appropriate bands for use in the classification algo- 
rithms are described. The chapter concludes with a dis- 
cussion of object-based image analysis (OBIA) 
procedures that are used to extract homogeneous 
patches (polygons) of information from the remote 


sensor data instead of thematic information in pixel 
format. Nonmetric methods including neural networks 
and expert system decision-tree classifiers are intro- 
duced in Chapter 10. The algorithms required to ex- 
tract information from hyperspectral imagery are 
discussed in Chapter 11. 


Ne Introduction 


Pattern recognition is normally applied to multispec- 
tral remote sensor data that has been geometrically 
corrected. General steps required to extract land-cover 
information from digital multispectral remote sensor 
data are summarized in Figure 9-1. 


Multispectral classification may be performed using a 
variety of methods, including (Figures 9-1 and 9-2): 


¢ algorithms based on parametric and nonparametric 
statistics that use ratio- and interval-scaled data and 
nonmetric methods that can also incorporate nomi- 
nal scale data, 


¢ the use of supervised or unsupervised classification 
logic, 

¢ use of hard (crisp) or soft (fuzzy) classification logic, 

« use of per-pixel or object-based classification logic, 
and 

¢ hybrid approaches. 


Parametric methods such as maximum likelihood clas- 
sification and unsupervised clustering assume normal- 
ly distributed remote sensor data and knowledge about 
the forms of the underlying class density functions 
(Duda et al., 2001). Nonparametric methods such as 
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nearest-neighbor classifiers, fuzzy classifiers, and neu- 
ral networks may be applied to remote sensor data that 
are not normally distributed and without the assump- 
tion that the forms of the underlying densities are 
known. Nonmetric methods such as rule-based deci- 
sion-tree classifiers can operate on both real-valued 
data (e.g., reflectance values from 0 to 100%) and nom- 
inal scaled data (e.g., class 1 = forest; class 2 = agricul- 
ture) (e.g., Jensen et al., 2009; Myint et al., 2011). 


In a supervised classification, the identity and location 
of some of the land-cover types (e.g., urban, agricul- 
ture, or wetland) are known a priori (i.e., before) 
through a combination of fieldwork, interpretation of 
aerial photography, map analysis, and personal experi- 
ence. The analyst attempts to locate specific sites in the 
remotely sensed data that represent homogeneous ex- 
amples of these known land-cover types. These areas 
are commonly referred to as training sites because the 
spectral characteristics of these known areas are used 
to train the classification algorithm. Multivariate sta- 
tistical parameters (means, standard deviations, covari- 
ance matrices, correlation matrices, etc.) are calculated 
for each training site. Every pixel both within and out- 
side the training sites is then evaluated and assigned to 
the class of which it has the highest likelihood of being 
a member. 


In an unsupervised classification, the identities of land- 
cover types to be specified as classes within a scene are 
not generally known a priori because ground reference 
information is lacking or surface features within the 
scene are not well defined. The computer is required to 
group pixels with similar spectral characteristics into 
unique clusters according to some statistically deter- 
mined criteria. The analyst then relabels and combines 
the spectral clusters into information classes. 


Supervised and unsupervised classification algorithms 
typically use hard (crisp) classification logic to produce 
a classification map that consists of hard, discrete cate- 
gories (e.g., forest, agriculture) (Figure 9-2a). Con- 
versely, it is also possible to use fuzzy (soft) 
classification logic, which takes into account the het- 
erogeneous and imprecise nature of the real world 
(Figure 9-2b). Fuzzy classification produces thematic 
output products that contain fuzzy information. Fuzzy 
classification is based on the fact that remote sensing 
detectors record the reflected or emitted radiant flux 
from heterogeneous mixtures of biophysical materials 
such as soil, water, and vegetation found within the IF- 
OV. The land-cover classes found within the [FOV 
(pixel) often grade into one another without sharp, 
hard boundaries. Thus, reality is actually very impre- 
cise and heterogeneous; that is, it is fuzzy (Jensen et al., 
2009). Instead of being assigned to just a single class 
out of m possible classes, each pixel in a fuzzy classifi- 


cation has m membership grade values that describe 
the proportion of the m land-cover types found within 
the pixel (e.g., 10% bare soil, 10% scrub-shrub, 80% 
forest) (Figure 9-2b). This information may be used to 
extract more precise land-cover information, especially 
concerning the makeup of mixed pixels (Foody, 2002). 


In the past, most digital image classification was based 
on processing the entire scene pixel by pixel. This is 
commonly referred to as_per-pixel classification 
(Blaschke and Strobl, 2001; Myint et al., 2011). Geo- 
graphic object-based image analysis (GEOBIA) tech- 
niques allow the analyst to decompose the scene into 
many relatively homogenous image objects (referred 
to as patches or segments) using a multiresolution im- 
age segmentation process (Blaschke et al., 2014). The 
various statistical characteristics of these homoge- 
neous image objects in the scene are then subjected to 
traditional statistical or fuzzy logic classification. Ob- 
ject-based classification based on image segmentation 
is often used for the analysis of high spatial resolution 
imagery (e.g., GeoEye-1, Pleiades, WorldView-2). 


No pattern classification method is inherently superior to 
any other. The nature of the classification problem, the 
biophysical characteristics of the study area, the distri- 
bution of the remotely sensed data (e.g., normally dis- 
tributed), and a priori knowledge determine which 
classification algorithm will yield useful results (Duro 
et al., 2012). Duda et al. (2001) provide sound advice: 
“We should have a healthy skepticism regarding studies 
that purport to demonstrate the overall superiority of a 
particular learning or recognition algorithm.” 


We Supervised Classification 


Useful thematic information may be obtained using su- 
pervised classification algorithms if the general steps 
summarized in Figure 9-1 are understood and applied. 
The analyst first specifies the geographic region of in- 
terest (ROT) on which to test hypotheses. The classes of 
interest to be examined are then carefully defined in a 
classification scheme. The classes of interest normally 
dictate whether the analyst should produce hard or 
fuzzy output products and whether per-pixel or object- 
oriented classification logic should be used. Next, the 
analyst obtains the appropriate digital remote sensor 
data, keeping in mind both sensor system and environ- 
mental constraints. Ideally ground reference informa- 
tion is obtained at the same as the remote sensing data 
acquisition. The remote sensor data are radiometrical- 
ly and geometrically corrected as discussed in Chapters 
6 and 7. An appropriate classification algorithm is then 
selected and the analyst selects initial training data (if 
necessary). Feature (band) selection is performed to 
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General Steps Used to Extract Thematic Land-Cover 
Information from Digital Remote Sensor Data 


State the nature of the land-cover classiXcation problem. 
* Specify the geographic region of interest (ROI). 
* DeXine the classes of interest within a classiXcation scheme. 
* Determine if it is to be a hard (crisp) or soft (fuzzy) classiXcation. 
* Determine if it is to be a per-pixel or object-based (OBIA) classiXcation. 
Acquire appropriate remote sensing and initial ground reference data. 
* Select remotely sensed data based on the following criteria: 
- Remote sensing system considerations: 
- Spatial, spectral, temporal, and radiometric resolution 
- Environmental considerations: 
- Atmospheric, soil moisture, phenological cycle, etc. 
* Obtain initial ground reference data based on: 
- a priori knowledge of the study area 
Process remote sensor data to extract thematic information. 
* Radiometric correction (or normalization) if necessary (Chapter 6) 
* Geometric correction (Chapter 7) 
* Select appropriate image classiXcation logic: 
- Parametric (e.g., maximum likelihood, clustering) 
- Nonparametric (e.g., nearest-neighbor, neural network) 
- Nonmetric (e.g., rule-based decision-tree classiXer) 
* Select appropriate image classiXcation algorithm: 
- Supervised, e.g., 
- Parallelepiped, minimum distance, maximum likelihood 
- Others (hyperspectral matched Kltering, spectral 
angle mapper — Chapter 11) 
- Unsupervised, e.g., 
- Chain method, multiple-pass ISODATA 
- Others (fuzzy c-means) 
- Hybrid involving artiXcial intelligence (Chapter 10), e.g., 
- Expert system decision-tree, neural network, support 
vector machine 
* Extract data from initial training sites (if required). 
* Select the most appropriate bands using feature selection criteria: 
- Graphical (e.g., two-dimensional feature space plots) 
- Statistical (e.g., transformed divergence, TM-distance) 
* Extract training statistics and rules based on: 
- Final band selection (if required), and/or 
- Machine-learning (Chapter 10) 
* Extract thematic information: 
- For each pixel or for each OBIA image object (supervised) 
- Label pixels or image objects (unsupervised) 
Accuracy assessment (Chapter 13). 
* Select method: 
- Qualitative conXidence-building 
- Statistical measurement 
* Determine number of samples required by class. 
* Select sampling scheme. 
* Obtain ground reference test information. 
* Create and analyze error matrix: 
- Univariate and multivariate statistical analysis. 
Accept or reject previously stated hypothesis. 
Distribute results if accuracy is acceptable. 


FIGURE 9-1 The general steps used to extract thematic land-cover information from digital remote sensor data. 
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ClassiXcation of Remotely Sensed Data Based on Hard (Crisp) versus Soft (Fuzzy) Logic 


Single-Stage Hard ClassiXcation 
of ® e Pixel to One Class 


1 Rectiled 
remotely 
sensed data 


Hard partition of feature 
space and assignment of 
each pixel to one of m 
classes using supervised 
and/or unsupervised 
classiication logic. 


Final classication map 
consisting of mm classes. 


a. 
b. 


Computation of Fuzzy Membership 
Grades and Final ClassiXcation 


1 Rectifled 
3 remotely 


in sensed data 


Fuzzy partition of feature space 
where each pixel has a membership 
grade value (from 0 to 1) for m classes 
using supervised and/or unsupervised 
classiXcation logic. 


DAnBWNre 


~ 
3 
= 


Application of additional 
logic to the membership 
grade information to derive 
a Mnal classiXcation map 
consisting of m classes, if 
desired. 


FI@ RE 9-2 Relationship between a traditional single-stage hard (crisp) classification using supervised or unsupervised clas- 
sification logic and a classification based on the use of soft (fuzzy) logic. 


determine the multispectral bands optimal for discrim- 
inating one training class from another. Additional 
training data are collected (if necessary) and the classi- 
fication algorithm is applied, yielding a classification 
map. A rigorous accuracy assessment (often referred to 
as an error evaluation) is then performed (see Chapter 
13). The classification maps and associated statistics 
are then distributed to colleagues and agencies when 
the results are acceptable. 


Land-Use and Land-Cover 
Classification Schemes 


Land cover refers to the type of material present on the 
landscape (e.g., water, crops, forest, wetland, human- 
made materials such as asphalt). Land use refers to 
what people do on the land surface (e.g., agriculture, 
commerce, settlement). The pace, magnitude, and scale 
of human alterations of the Earth’s land surface are 
unprecedented in human history. Consequently, land- 
cover and land-use data are central to such United Na- 
tions’ Agenda 2] issues as combating deforestation, 
managing sustainable settlement growth, and protect- 
ing the quality and supply of water resources (Jensen et 
al., 2002). In light of the human impacts on the land- 
scape, there is a need to establish baseline datasets 
against which changes in land cover and land use can 
be assessed (Warner et al., 2009; Weng, 2014). 


The International Geosphere-Biosphere Programme 
(IGBP) and the International Human Dimensions of 
Global Environmental Change Programme (IHDP) 
suggest that: 


over the coming decades, the global effects of 
land use and cover change may be as significant, 
or more so, than those associated with potential 
climate change. Unlike climate change per se, 
land use and cover change are known and un- 
disputed aspects of global environmental 
change. These changes and their impacts are 
with us now, ranging from potential climate 
warming to land degradation and biodiversity 
loss and from food production to spread of in- 
fectious diseases (IGBP-IHDP, 2002). 


Land-cover data have proved especially valuable for 
predicting the distribution of both individual species 
and species assemblages across broad areas that could 
not otherwise be surveyed. Various predictive models 
have gained currency as the availability and accuracy 
of land-cover datasets have improved. For example, re- 
mote sensing—derived land-cover information is used 
extensively in the Gap Analysis Program (GAP), which 
is the largest species distribution modeling effort. The 
goal is to develop detailed maps of habitat preferences 
for target species and monitor plant phenology (Kerr 
and Ostrovsky, 2003). 
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All classes of interest must be selected and defined 
carefully to classify remotely sensed data successfully 
into land-use and/or land-cover information (Congal- 
ton and Green, 2009). This requires the use of a classi- 
fication scheme containing taxonomically correct 
definitions of classes of information that are organized 
according to logical criteria. If a hard (crisp) classifica- 
tion is to be performed, then the classes in the classifi- 
cation system should normally be: 


* mutually exclusive, 
¢ exhaustive, and 
¢ hierarchical. 


Mutually exclusive means that there is no taxonomic 
overlap (or fuzziness) of any classes (i.e., deciduous 
forest and evergreen forest are distinct classes). Ex- 
haustive means that all land-cover classes present in the 
landscape are accounted for and none has been omit- 
ted. Hierarchical means that sublevel classes (e.g., sin- 
gle-family residential, multiple-family residential) may 
be hierarchically combined into a higher-level category 
(e.g., residential) that makes sense. This allows simpli- 
fied thematic maps to be produced when required. 


It is also important for the analyst to realize that there 
is a fundamental difference between information class- 
es and spectral classes. Information classes are those 
that human beings define. Conversely, spectral classes 
are those that are inherent in the remote sensor data 
and must be identified and then labeled by the analyst. 
For example, in a remotely sensed image of an urban 
area there is likely to be single-family residential hous- 
ing. A relatively coarse spatial resolution remote sen- 
sor such as SPOT (20 X 20 m) might be able to record a 
few relatively pure pixels of vegetation and a few pure 
pixels of concrete/asphalt road or asphalt shingles. 
However, it is more likely that in this residential area 
the pixel brightness values will be a function of the re- 
flectance from mixtures of vegetation, asphalt and 
concrete. Few planners or administrators want to see a 
map labeled with classes like (1) concrete, (2) asphalt, 
(3) vegetation, and (4) mixture of vegetation and con- 
crete/asphalt (unless they are interested in a map of im- 
pervious surfaces). Rather, they typically prefer the 
analyst to relabel the mixture class as single-family res- 
idential. The analyst should do this only if in fact there 
is a good association between the mixture class and 
single-family residential housing. Thus, we see that an 
analyst must often translate spectral classes into infor- 
mation classes to satisfy bureaucratic requirements. An 
analyst should understand well the spatial and spectral 
characteristics of the sensor system and be able to re- 
late these system parameters to the types and propor- 
tions of materials found within the scene and within 
pixel IFOVs. If these parameters and relationships are 


understood, spectral classes often can be thoughtfully 
relabeled as information classes. 


Several hard classification schemes have been devel- 
oped that can readily incorporate land-use and/or 
land-cover data obtained by interpreting remotely 
sensed data. Only a few of these classification schemes 
will be discussed here, including: 


¢ American Planning Association (APA) Land-Based 
Classification Standard, which is oriented toward 
detailed land-use classification (Figure 9-3); 

* U.S. Geological Survey Land-UselLand-Cover Clas- 

ification System for Use with Remote Sensor Data; 

¢ US. National Land Cover Dataset (NLCD) Classi- 

fication scheme (Figure 9-4); 

« NOAA Coastal Change Analysis Program (C-CAP) 

classification scheme; 

¢ U.S. Department of the Interior Fish & Wildlife Ser- 
vice, Classification of Wetlands and Deepwater Habi- 
tats of the United States (Figures 9-5 and 9-6); 

¢ US. National Vegetation & Classification Standard; 

¢ International Geosphere-Biosphere Program JGBP 
Land Cover Classification System specially modified 
for the creation of MODIS land-cover products 
(Figure 9-7). 


a 


American Planning Association Land-Based 
Classification Standard (LBCS) 


Few classification schemes attempt to classify land use. 
In fact, most explicitly state that they are concerned 
only with land-cover information. Therefore, if the user 
is most interested in extracting detailed urban/subur- 
ban land-use information from relatively high spatial 
resolution remote sensor data, then one of the most 
practical and comprehensive hierarchical classification 
systems is the Land-Based Classification Standard 
(LBCS) developed by the American Planning Associa- 
tion (2014a). 


The LBCS requires input from in situ surveys, and/or 
aerial photography, and satellite remote sensor data to 
obtain information at the parcel level on the following 
five characteristics: activity, function, site development, 
structure, and ownership (American Planning Associa- 
tion, 2014a). The system provides a unique code and 
description for almost every commercial and industrial 
land-use activity (APA, 2014b). An example of the 
Land-Based Classification Standard applied to three 
parcels of land in Beaufort, SC, in 2011, is shown in 
Figure 9-3 along with their associated LBCS activity, 
function, and structure codes. Two of the commercial 
establishments are full-service fast-food restaurants 
with drive-throughs and the other is a furniture store. 


366 INTRODUCTORY DIGITAL IMAGE PROCESSING 


Ane rican Planning Association Land-Based Classification Standard (LBCS) 


Table 1: Activity 


Parcel ID Activity 
0171 2210 
0170 2100 
125A 2210 


furniture 


Table 3: Structural Character 


Parcel ID 


0171 2220 


0170 2592 
125A 2220 


Ie scription 
restaurant with drive-through 


restaurant with drive-through 


Structure 


Table 2: Function 


Function Le scription 


Te scription 


restaurant building 
home improvement center 
restaurant building 


FIGJ RE 9-3 The American Planning Association developed the Land-Based Classification Standard (LBCS) that contains de- 
tailed definitions of urban/suburban land use. The system incorporates information derived in situ and using remote sensing 
techniques. This is an oblique aerial photograph of three commercial buildings in Beaufort, SC, in 2011. Activity, function, and 
structural character codes associated with these three parcels are identified. Site development and ownership information at- 
tribute tables are not shown (aerial photography and parcel information courtesy of Beaufort County GIS Department). 


The LBCS is always under development. Users are en- 
couraged to keep abreast of the LBCS and to use it for 
very intensive urban/suburban studies that require de- 
tailed commercial and industrial land-use classification 
codes. The LBCS does not provide information on 
land-cover or vegetation characteristics in the urban 
environment; it relies on the Federal Geographic Data 
Committee standards on this topic. 


U.S. Geological Survey Anderson Land-Use/ 
Land-Cover Classification System for Use with 
Remote Sensor Data 


The U.S. Geological Survey’s Land-Use/Land-Cover 
Classification System for Use with Remote Sensor Data 


(Anderson et al., 1976) is primarily a resource-oriented 
land-cover classification system in contrast with people 
or activity land-use classification systems such as the 
APA’s Land-Based Classification Standard. The USGS 
rationale is that “although there is an obvious need for 
an urban-oriented land-use classification system, there 
is also a need for a resource-oriented classification sys- 
tem whose primary emphasis would be the remaining 
95 percent of the United States land area.” The USGS 
system addresses this need with eight of the nine origi- 
nal Level I categories that treat land area that is not in 
urban or built-up categories (Table 9-1). The system is 
designed to be driven primarily by the interpretation of 
remote sensor data obtained at various scales and reso- 
lutions (Table 9-2) and not data collected in situ. The 
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TABLE 9-1 U.S. Geological Survey Land-Use/Land-Cover 
Classification System for Use with Remote Sensor Data 
(Anderson et al., 1976). 


Classification Level 


1 Urban or Built-up Land 

11 Residential 

12 Commercial and Services 

13 Industrial 

14 Transportation, Communications, and Utilities 
15 Industrial and Commercial Complexes 

16 Mixed Urban or Built-up 

17 Other Urban or Built-up Land 


2 Agricultural Land 

21 Cropland and Pasture 

22 Orchards, Groves, Vineyards, Nurseries, and Ornamental 
Horticultural Areas 

23 Confined Feeding Operations 

24 Other Agricultural Land 


3 Rangeland 

31 Herbaceous Rangeland 

32 Shrub-Brushland Rangeland 
33 Mixed Rangeland 


4 Forest Land 

41 Deciduous Forest Land 
42 Evergreen Forest Land 
43 Mixed Forest Land 


5 Water 

51 Streams and Canals 
52 Lakes 

53 Reservoirs 

54 Bays and Estuaries 


6 Wetland 
61 Forested Wetland 
62 Nonforested Wetland 


7 Barren Land 

71 Dry Salt Flats 

72 Beaches 

73 Sandy Areas Other Than Beaches 

74 Bare Exposed Rock 

75 Strip Mines, Quarries, and Gravel Pits 
76 Transitional Areas 

77 Mixed Barren Land 


8 Tundra 

81 Shrub and Brush Tundra 
82 Herbaceous Tundra 

83 Bare Ground Tundra 

84 Wet Tundra 

85 Mixed Tundra 


9 Perennial Snow or Ice 
91 Perennial Snowfields 
92 Glaciers 


classification system continues to be modified to sup- 
port a great variety of land-cover mapping activities by 
the U.S. Geological Survey, the Environmental Protec- 
tion Agency, the NOAA Coastal Services Center, the 
Multi-Resolution Land Characteristics consortium, 
and others. 


TABLE 9-2 Four levels of the U.S. Geological Survey Land- 
Use/Land-Cover Classification System for Use with Remote 
Sensor Data and the type of remotely sensed data typically 
used to provide the information (Anderson et al., 1976; Jen- 
sen and Cowen, 1999). 


Classification 


Level Typical Data Characteristics 


I Satellite imagery such as NOAA AVHRR 
(1.1% 1.1 km), MODIS (250 & 250 m; 

500 X 500 m), Landsat MSS (79 X 79 m), 
Landsat TM and Landsat 8 (30 X 30 m), and 
SPOT XS (20 W 20 m). 


Il Satellite imagery such as SPOT HRV multi- 
spectral (10 10 m) and Indian IRS 1-C 
panchromatic (5 & 5 m). High-altitude aeri- 
al photography acquired at scales smaller 
than 1:80,000. 


ll Satellite imagery with 141m to 2.5 2.5 
m nominal spatial resolution such as IKO- 

NOS. Medium-altitude aerial photography 
at scales from 1:20,000 to 1:80,000. 


IV Satellite imagery with < 1 1 m nominal 
spatial resolution (e.g., GeoEye-1; World- 
View-2). Low-altitude aerial photography 
at scales from 1:4,000 to 1:20,000 scale. 


National Land Cover Database (NLCD) 
Classification System 


The Multi-Resolution Land Characteristics (MRLC) 
consortium, a group of federal agencies led by the U.S. 
Geological Survey, originally joined forces in 1992 to 
use Landsat Thematic Mapper 30 X% 30 m imagery to 
create a “wall-to-wall” land-cover dataset of the nation 
called the National Land Cover Database (NLCD). The 
classification scheme used by the NLCD was adapted 
from the Anderson Land Cover Classification System 
and is listed in Table 9-3 (MRLC, 2014). The NLCD 
consists of three major data releases based on a 10-year 
production cycle: 


* acirca 1992 conterminous U.S. land-cover dataset 
(referred to as NLCD 1992), 


* a circa 2001 land-cover database for the U.S. and 
Puerto Rico (referred to as NLCD 2001), and 


* a 1992/2001 Land Cover Change Retrofit (LCCR) 
product. 


The NLCD 2006 represents the first database pro- 
duced with a 5-year repeat cycle and with additional 
products specifically designed for land-cover monitor- 
ing (e.g., percent impervious surface and percent tree 
canopy cover). NLCD 2011 is scheduled for release in 
2014 (Homer et al., 2012). The NLCD program has 
changed its emphasis from mapping to monitoring to 
address the emerging issues of sustainable use that are 
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TABLE 9-3 National Land Cover Dataset (NLCD) 2006 Classification Scheme (Homer et al., 2012; MRLC, 2014). The list does 
not include several Alaskan land-cover classes, including: dwarf scrub, sedge/herbaceous, lichens, and moss. 


[am 
——— Open Water — areas of open water, generally with <25% cover of vegetation or soil. 
[a [Fee on-set gy TTC 


Developed areas with a high percentage (>30%) of constructed materials (e.g. asphalt, concrete, buildings, etc.). 
Developed, Open Space — a mixture of constructed materials, but mostly vegetation in the form of lawn grasses. 


Impervious surfaces account for <20% of total cover. Includes large-lot single-family housing units, parks, golf 
courses, and vegetation planted in developed settings for recreation, erosion control, or aesthetic purposes. 


Developed, Low Intensity — areas with a mixture of constructed materials and vegetation. Impervious surfaces 
account for 20% to 49% of total cover. These areas most commonly include single-family housing units. 


Developed, Medium Intensity — areas with a mixture of constructed materials and vegetation. Impervious sur- 
faces account for 50% to 79% of the total cover. These areas most commonly include single-family housing units. 


Developed High Intensity — highly developed areas where people reside or work in high numbers such as apart- 
ments, row houses and commercial/industrial. Impervious surfaces account for 80% to 100% of the total cover. 


Barren characterized by bare rock, gravel, sand, silt, clay, or other earthen material, with little or no “green” vegetation 
present regardless of its inherent ability to support life. Vegetation, if present, is more widely spaced and scrub- 
by than that in the green vegetated categories; lichen cover may be extensive. 


Barren Land (Rock/Sand/Clay) — bedrock, desert pavement, scarps, talus, slides, volcanic material, glacial debris, 
sand dunes, strip mines, gravel pits and other earthen material. Vegetation accounts for <15% of total cover. 


Dwarf Scrub — Alaska only areas dominated by shrubs <20 cm tall with shrub en typically >20% of total veg- 
ar vegetation. 


etation. This type is often co-associated with grasses, sedges, herbs, and non-vascu 


Shrub/Scrub — areas dominated by shrubs; <5 m tall with shrub canopy typically >20% of total vegetation. Class 
includes true shrubs, young trees in an early successional stage or trees stunted from environmental conditions. 


characterized by natural or semi-natural woody vegetation with aerial stems, generally <6 m tall, with individuals 
or clumps not touching to interlocking. Both evergreen and deciduous species of true shrubs, young trees, and 
trees or shrubs that are small or stunted because of environmental conditions are included. 


characterized by natural or semi-natural herbaceous vegetation that accounts for 75% to 100% of cover. 


71 | Grassland/Herbaceous — areas dominated by gramanoid or herbaceous vegetation, generally >80% of total 
vegetation. These areas are not subject to intensive management such as tilling, but can be utilized for grazing. 


Planted/Cultivated characterized by herbaceous vegetation that has been planted or is intensively managed for the production of 
food, feed, or fiber. Herbaceous vegetation accounts for 75% to 100% of the cover. 
Pasture/Hay — grasses, legumes, or grass-legume mixtures planted for livestock grazing or the production of 
seed or hay crops, typically on a perennial cycle. Pasture/hay vegetation accounts for >20% of total vegetation. 
82 | Cultivated Crops — areas used for the production of annual crops, such as corn, soybeans, vegetables, tobacco, 
and cotton, an pee erential woody crops such as orchards and elas Crop vegetation accounts for >20% 
of total vegetation. This class also includes all land being actively tilled. 


where the soil or substrate is periodically saturated or covered with water as defined by Cowardin et al. (1979). 


Woody Wetlands — areas where forest or shrubland vegetation accounts for >20% of vegetative cover and the 
soil or substrate is periodically saturated with or covered with water. 


ae Emergent Herbaceous Wetlands — Areas where perennial herbaceous vegetation accounts for >80% of vegeta- 


tive cover and the soil or substrate is periodically saturated with or covered with water. 
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2006 National Land Cover Dataset (NLCD) 


NLCD Classification Scheme 


11 Open Water 

|__| 12 Perennial Ice/Snow 

21 Developed, Open Space 

22 Developed, Low Intensity 

23 Developed, Medium Intensity 
24 Developed, High Intensity 

31 Barren Land (Rock, Sand, Clay) 
41 Deciduous Forest 

42 Evergreen Forest 

43 Mixed Forest 

51 Dwarf Scrub* 

52 Shrub/Scrub 

71 Grassland/Herbaceous 

72 Sedge/Herbaceous* 

73 Lichens* 

74 Moss* 

81 Pasture/Hay 

82 Cultivated Crops 

90 Woody Wetlands 

95 Emergent Herbaceous Wetlands 


* Alaska only 


Gein as 


ao Ie sehee PLL. ee tee. 


FIGURE 9-4 a) 2006 NLCD land cover information of the conterminous United States. b) 2006 NLCD land-cover information 
of South Carolina. c) 2006 NLCD land-cover information for Columbia, SC (NLCD information courtesy of the Multi-Resolution 
Land Characteristics (MRLC) consortium). 


faced by the MRLC and its user community (Xian et — state of South Carolina, and Columbia, SC, are dis- 
al., 2009; Fry et al., 2011). NLCD 2006 land-cover in- played in Figure 9-4. 
formation for the conterminous United States, the 
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TABLE 9-4 NOAA Coastal Change Analysis Program (C-CAP) land cover classification scheme (NOAA C-CAP, 2014). 


Class/Value 


NOAA C-CAP Land-Cover Classification Description 


Developed 

Forest 

Shrubland 
Herbaceous 
Planted/Cultivated 


Identical to NLCD classes listed in Table 9-3. 


Palustrine Wetlands 


13 


Palustrine Forested Wetland — includes tidal and nontidal wetlands dominated by woody vegetation >5 m in 
height, and all such wetlands that occur in tidal areas in which salinity due to ocean-derived salts is <0.5%. Total 
vegetation coverage is >20%. 


14 


Palustrine Scrub/Shrub Wetland - includes tidal and nontidal wetlands dominated by woody vegetation <5 m in 
height, and all such wetlands that occur in tidal areas in which salinity due to ocean-derived salts is <0.5%. Total 
vegetation coverage is >20%. Species present could be true shrubs, young trees and shrubs, or trees that are 
small or stunted due to environmental conditions. 


15 


Palustrine Emergent Wetland (Persistent) — includes tidal and nontidal wetlands dominated by persistent emer- 
gent vascular plants, emergent mosses or lichens, and all such wetlands that occur in tidal areas in which salinity 
due to ocean-derived salts is <0.5%. Total vegetation cover is >80%. Plants generally remain standing until the 
next growing season. 


Estuarine Wetlands 


16 


Estuarine Forested Wetland — includes tidal wetlands dominated by woody vegetation >5 m in height, and all 
such wetlands that occur in tidal areas in which salinity due to ocean-derived salts is >0.5%. Total vegetation cov- 
erage is >20%. 


Alz/ 


Estuarine Scrub / Shrub Wetland — includes tidal wetlands dominated by woody vegetation <5 m in height, and 
all such wetlands that occur in tidal areas in which salinity due to ocean-derived salts is >0.5%. Total vegetation 
coverage is >20%. 


18 


Estuarine Emergent Wetland — Includes all tidal wetlands dominated by erect, rooted, herbaceous hydrophytes 
(excluding mosses and lichens). Wetlands that occur in tidal areas in which salinity due to ocean-derived salts is 
>0.5% and that are present for most of the growing season in most years. Total vegetation cover is >80%. Peren- 
nial plants usually dominate these wetlands. 


Barren Land 


19 


Unconsolidated Shore — includes material such as silt, sand, or gravel that is subject to inundation and redistribu- 
tion due to the action of water. Substrates lack vegetation except for pioneering plants that become established 
during brief periods when growing conditions are favorable. 


Water and 
Submerged Lands 


21 | Open Water - include areas of open water, generally with <25% cover of vegetation or soil. 

22 | Palustrine Aquatic Bed — includes tidal and nontidal wetlands and deepwater habitats in which salinity due to 
ocean-derived salts is <0.5% and which are dominated by plants that grow and form a continuous cover princi- 
pally on or at the surface of the water. These include algal mats, detached floating mats, and rooted vascular 
plant assemblages. Total vegetation cover is >80%. 

23 | Estuarine Aquatic Bed — includes tidal wetlands and deepwater habitats in which salinity due to ocean-derived 


salts is >0.5% and which are dominated by plants that grow and form a continuous cover principally on or at the 
surface of the water. These include algal mats, kelp beds, and rooted vascular plant assemblages. Total vegeta- 
tion cover is >80%. 


It is important to note that the NLCD classification 
scheme provides some detailed level IT rural land cover 
information (e.g., three forest classes, two wetland 
classes, two agricultural classes). However, the classifi- 
cation scheme does not differentiate between residen- 
tial, commercial and _ services, industrial, or 
transportation land cover as in the Anderson classifi- 
cation system. Rather, the emphasis in urbanized areas 
is on the identification of barren land and low-, medi- 
um-, and high-density development. This makes it 
somewhat difficult to compare NLCD urban/suburban 


land-cover information with more traditional land cov- 
er studies that differentiate between residential, com- 
mercial, industrial, and transportation land cover. 


NOAA Coastal Change Analysis Program 
(C-CAP) Classification Scheme 


NOAA’s Coastal Change Analysis Program (C-CAP) 
produces a nationally standardized database of land- 
cover and land-change information for the coastal re- 
gions of the U.S (Table 9-4). In fact, C-CAP provides 
the “coastal expression” information for the NLCD 
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(NOAA C-CAP, 2014). C-CAP products provide in- 
ventories of coastal intertidal areas, wetlands, and ad- 
jacent uplands with the goal of monitoring these 
habitats by updating the land-cover maps every five 
years. C-CAP products are developed using multiple 
dates of remotely sensed imagery and consist of raster- 
based land-cover maps for each date of analysis, as well 
as a file that highlights what changes have occurred be- 
tween these dates and where the changes were located 
(NOAA C-CAP, 2014). 


The NOAA C-CAP classification system includes 
many of the same classes as the NLCD but provides 
very detailed information about wetland-related phe- 
nomena, especially in the coastal zone. Some of the 
class definitions come from the Cowardin wetland clas- 
sification system discussed below. NOAA provides an 
online C-CAP Land Cover Atlas where the results of C- 
CAP land-cover mapping and change detection can be 
viewed (http://www.csc.noaa.gov/ccapatlas/). 


U.S. Department of the Interior Fish & Wildlife 
Service Classification of Wetlands and 
Deepwater Habitats of the United States 


The conterminous U.S. continues to lose inland and 
coastal wetland to agricultural, residential, and com- 
mercial land-use development. Therefore, there is great 
interest in wetland mapping. Detenbeck (2002) and 
EPA (2008) review numerous methods used to classify 
wetlands and the characteristics of several wetland 
classification schemes. 


In the U.S., the Department of the Interior Fish & 
Wildlife Service is responsible for mapping and inven- 
torying wetland. Therefore, they developed a wetland 
classification system that incorporates information ex- 
tracted from remote sensor data and in situ measure- 
ment (Cowardin et al., 1979). Commonly referred to as 
the Cowardin System, it describes ecological taxa, ar- 
ranges them in a system useful to resource managers, 
and provides uniformity of concepts and terms. Wet- 
lands are classified based on plant characteristics, soils, 
and frequency of flooding. Ecologically important ar- 
eas of deep water, traditionally not considered wet- 
lands, are included in the classification as deep-water 
habitats. 


Five systems form the highest level of the classification 
hierarchy: marine, estuarine, riverine, lacustrine, and 
palustrine (Figure 9-5). Marine and estuarine systems 
each have two subsystems: subtidal and intertidal. The 
riverine system has four subsystems: tidal, lower peren- 
nial, upper perennial, and intermittent. The lacustrine 
has two, littoral and limnetic, and the palustrine has no 
subsystem. Within the subsystems, classes are based on 
substrate material and flooding regime or on vegetative 
life form. The same classes may appear under one or 


more of the systems or subsystems. The distinguishing 
features of the Riverine system are presented in Figure 
9-6a. Use of the classification system to characterize an 
Estuarine salt marsh near Bluffton, SC, is shown in 
Figure 9-6b. It highlights the characteristics of Tall 
Creekside and Intermediate Spartina alterniflora. This 
salt-tolerant habitat dominates much of the estuarine 
habitat along the eastern seaboard. 


The Cowardin system was the first nationally recog- 
nized wetland classification scheme. It was adopted as 
the National Vegetation Classification Standard for 
wetlands mapping and inventory by the Wetlands Sub- 
committee of the Federal Geographic Data Commit- 
tee. The Cowardin wetland classification system is the 
most practical scheme to use if you are going to extract 
wetland information from remotely sensed data and 
share the information with others interested in wet- 
land-related problems. However, it is important to note 
that there are several other wetland classification 
schemes available (Detenbeck, 2002; EPA, 2008; 
NOAA C-CAP, 2014). 


U.S. National Vegetation Classification 
Standard (NVCS) 


The Vegetation Subcommittee of the U.S. Federal 
Geographic Data Committee has endorsed the Nation- 
al Vegetation Classification Standard (NVCS), which 
produces uniform vegetation resource data at the na- 
tional level (FGDC, 2014). Separate categories are pro- 
vided for natural and cultural vegetation, consistent 
with many other vegetation and land-cover classifica- 
tion schemes: 


* Natural vegetation is defined as vegetation where 
ecological processes primarily determine species and 
site characteristics; that is, vegetation comprised of 
a largely spontaneously growing set of plant species 
that are shaped by both site and biotic processes. 
Natural vegetation forms recognizable physiog- 
nomic and floristic groupings that can be related to 
ecological site features. 


* Cultural vegetation is defined as vegetation with a 
distinctive structure, composition, and develop- 
ment determined or influenced by regular human 
activity. The distinctive physiognomy, floristics, and 
dependence on human activity for its persistence set 
cultural vegetation apart from natural and/or semi- 
natural vegetation. 


The National Vegetation Classification Standard is a 
good place to begin if a scientist is interested in extract- 
ing detailed vegetation information from remote sen- 
sor data and placing it in an ecologically sound 
vegetation classification system. 
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FIGURE 9-5 The Classification of Wetlands and Deepwater Habitats of the United States hierarchy of wetland habitat sys- 
tems, subsystems, and classes (Cowardin et al., 1979). The palustrine system does not include deepwater habitats. The Cowar- 
din system is the National Vegetation Classification Standard for wetlands mapping and inventory (FGDC Wetland Mapping 
Standard, 2014). 
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Cross-section of Riverine Habitats in the Classification of Wetlands and Deepwater Habitats of the U.S. 


Upland pawns Riverine 


Palustrine Upland 
pa 


7 
k 
x 
v 


Bottom 
tland 


rs 
i 


Wi 


Unconsolidated 
Unconsolidated 
Emergent Wetland 
Nonpersistent 
Emergent Wetland 
Persistent 
Scrub-Shrub 


ici ae 


High Water EL é, 
s— Average Water— b 

a. Temporarily Mooded d Low Water —— 
b. Seasonally Kooded 


c. Semipermanently Xooded 


e d. Intermittently exposed 
e. Permanently Kooded 


Cross-section of Estuarine Intertidal Habitats in the Classification of Wetlands 
and Deepwater Habitats of the U.S. for a Salt Marsh near Bluffton, SC 
Palustrine Emergent Persistent Marsh 
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Tall G eekside Spartina alterniflora 
(Smooth Cordgrass) is more dense 
on the creekbanks where diurnal 
tidal Mushing deposits nutrients 
and maintains reasonable levels of 
salinity. Over time, the creekside 
Spartina may trap suspended 
sediments, resulting in a gradual 
increase in the elevation of the 
tidal creek berm. 


Intermediate Spartina 
alterniflora (Smooth 
Cordgrass) grows in 
a more stressed 
environment behind 
the creekbank, where 
nutrients are not as 
plentiful and there 
may be increased 
salinity levels in the 
substrate because 

of reduced tidal 
Mushing. 


FIGURE 9-6 a) Distinguishing features of habitats in the Riverine system of the Classification of Wetlands and Deepwater 
Habitats of the United States (Cowardin et al., 1979). b) Use of the classification scheme to characterize an Estuarine salt 


marsh near Bluffton, SC, along the May River (Jensen and Hodgson, 2011). A classification of this type of wetland habitat us- 
ing object-based image analysis (OBIA) is presented in Figure 9-38c. 
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TABLE 9-5 IGBP Land-Cover Classification System modified for the creation of MODIS Land-Cover Type products (based on 
NASA Earth Observatory, 2002; Friedl et al., 2002, 2010; Giri et al., 2005). 


Natural Vegetation 


Description 


Forest 

Evergreen Needleleaf 
Evergreen Broadleaf 
Deciduous Needleleaf 
Deciduous Broadleaf 


Mixed Forest 


Dominated by woody vegetation with a cover >60% and height >2 m. Almost all trees re- 
main green all year. Canopy is never without green foliage. 

Dominated by woody vegetation with a cover >60% and height >2 m. Most trees and 
shrubs remain green year round. Canopy is never without green foliage. 

Dominated by woody vegetation with a cover >60% and height >2 m. Seasonal needle- 
leaf communities with an annual cycle of leaf-on and leaf-off periods. 

Dominated by woody vegetation with a cover >60% and height >2 m. Broadleaf tree 
communities with an annual cycle of leaf-on and leaf-off periods. 

Dominated by trees with a cover >60% and height >2 m. Communities with interspersed 
mixtures or mosaics of the other four forest types. None of the forest types exceeds 
60% of landscape. 


Shrubland, Grasslands, and Wetland 
Closed Shrublands 

Open Shrublands 

Woody Savannas 

Savannas 

Grasslands 


Permanent Wetlands 


Woody vegetation <2 m tall and with shrub canopy cover >60%. The shrub foliage can be 
either evergreen or deciduous. 

Woody vegetation <2 m tall and with shrub canopy cover between 10% and 60%. The 
shrub foliage can be either evergreen or deciduous. 

Herbaceous and other understory systems, and with forest canopy cover between 30% 
and 60%. The forest cover height >2 m. 

Herbaceous and other understory systems, and with forest canopy cover between 10% 
and 30%. The forest cover height >2 m. 

Lands with herbaceous types of cover. Tree and shrub cover is <10%. 


Lands with a permanent mixture of water and herbaceous or woody vegetation. Vegeta- 
tion can be present in salt, brackish, or fresh water. 


Developed and Mosaic Lands 


Agriculture 


Croplands 


Lands covered with temporary crops followed by harvest and a bare-soil period (e.g., sin- 
gle and multiple cropping systems). Perennial woody crops are classified as the appropri- 
ate forest or shrub land-cover type. 


Cropland/Natural Vegetation Mosaic 


Cropland/Natural Vegetation Mosaic 


Mosaic of croplands, forests, shrubland, and grasslands in which no one component com- 
prises >60% of the landscape. 


Urban 
Built-up 


Land covered by buildings and other human-made structures. 


Nonvegetated Lands 


Barren 


Barren or Sparsely Vegetated 


Exposed soil, sand, rocks, or snow and <10% vegetated cover during the year. 


Snow and Ice 


Lands under snow/ice cover throughout the year. 


Water 


Oceans, seas, lakes, reservoirs, and rivers. Can be fresh or salt water. 


International Geosphere-Biosphere Program 
IGBP Land-Cover Classification System 


Land Cover Type Products (MCD12Q1) that are pro- 
duced each calendar year. These include: 


M dified for the Creation of MODIS Land- 


Cover Type Products 


If a scientist is interested in inventorying land cover at 


a 17-class IGBP land-cover classification; 
¢ al14-class University of Maryland classification; 


the regional, national, and global scale, then the modi- * a 10-class LAI/FPAR (leaf-area-index/fraction of 


fied International Geosphere-Biosphere Program 


photosynthetically active radiation) classification; 


Land-Cover Classification System may be appropriate. ¢ an 8-Biome classification; anda 


For example, Friedl et al. (2010) describe the MODIS 


e 12-class plant functional type classification. 
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Land Cover Map of North America Derived from Terra MODIS Data 
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FIGURE 9-7 a) Land-cover map of North America produced from Terra MODIS 1 & 1 km data collected between November 
2000 and October 2001. b) Classification legend (images courtesy of MODIS Land Cover and Land Cover Dynamics group at 
the Boston University Center for Remote Sensing, NASA Goddard Space Flight Center (2004) and NASA Earth Observatory 


(2002)). 


The characteristics of the IGBP classification scheme 
are found in Table 9-5. Collection 4 MODIS Global 
Land Cover Type (MLCT) products were produced at 
1-km resolution. Collection 5 MLCT products are gen- 
erated at 500-m spatial resolution. An example of Col- 
lection 4 MODIS land cover generated from images 


collected between November 2000 and October 2001 is 
shown in Figure 9-7. 


Observations about Classification Schemes 

Geographical information (including thematic maps 
and remote sensor data) is often imprecise. For exam- 
ple, there is usually a gradual transition at the interface 
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Nominal Spatial Resolution (m) 
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FIGURE 9-8 Nominal spatial resolution requirements as a 
function of the mapping requirements for Levels | to IV 
land-cover classes in the United States (based on Anderson 
et al., 1976). Note the dramatic increase in spatial resolu- 
tion required to map Level II classes (based on Welch, 1982; 
Jensen and Cowen, 1999). 


of forests and rangeland, yet many of the aforemen- 
tioned classification schemes insist on a hard boundary 
between the classes at this transition zone. The 
schemes should contain fuzzy definitions because the 
thematic information they contain is fuzzy (Wang, 
1990a). Fuzzy classification schemes are not currently 
standardized. They are typically developed by individ- 
ual researchers for site-specific projects. The fuzzy clas- 
sification systems developed may not be transferable to 
other environments. Therefore, we tend to see the use 
of existing hard classification schemes, which are rigid, 
based on a priori knowledge, and generally difficult to 
use. They continue to be widely employed because they 
are scientifically based and different individuals using 
the same classification system can compare results. 


This brings us to another important consideration. If a 
reputable classification system already exists, it is fool- 
ish to develop an entirely new system that will probably 
be used only by ourselves. It is better to adopt or mod- 
ify existing nationally or internationally recognized 
classification systems. This allows us to interpret the 
significance of our classification results in light of oth- 
er studies and makes it easier to share data. 


Finally, it should be reiterated that there is a general re- 
lationship between the level of detail in a classification 


scheme and the spatial resolution of remote sensor sys- 
tems used to provide information. Welch (1982) sum- 
marized this relationship for mapping urban and 
suburban land use and land cover in the United States 
(Figure 9-8). A similar relationship exists when map- 
ping vegetation (Botkin et al., 1984). For example, the 
sensor systems and spatial resolutions useful for dis- 
criminating vegetation from a global to an in situ per- 
spective are summarized in Figure 9-9. This suggests 
that the level of detail in the desired classification sys- 
tem dictates the spatial resolution of the remote sensor 
data that should be used. Of course, the spectral reso- 
lution of the remote sensing system is also an impor- 
tant consideration, especially when inventorying 
vegetation, water, ice, snow, soil, and rock. 


Training Site Selection and Statistics 
Extraction 


An image analyst may select training sites within the 
image that are representative of the land-cover or land- 
use classes of interest after the classification scheme is 
adopted (Friedl et al., 2010). The training data should 
be of value if the environment from which they were 
obtained is relatively homogeneous. For example, if all 
the soils in a grassland region are composed of well- 
drained sandy-loam soil, then it is likely that grassland 
training data collected throughout the region would be 
representative. However, if the soil conditions change 
across the study area (e.g., one-half of the region has a 
perched water table with moist near-surface soil), it is 
likely that grassland training data acquired in the dry- 
soil part of the study area will not be representative of 
the spectral conditions for grassland found in the 
moist-soil portion of the study area. Thus, we have a 
geographic signature extension problem, meaning that 
it may not be possible to extend our grassland remote 
sensing training data through x, y space. 


The easiest way to remedy this situation is to apply 
geographical stratification during the preliminary stag- 
es of a project. At this time all significant environmen- 
tal factors that contribute to geographic signature 
extension problems should be identified, such as differ- 
ences in soil type, water turbidity, crop species (e.g., 
two strains of wheat), unusual soil moisture conditions 
possibly caused by a thunderstorm that did not uni- 
formly deposit its precipitation, scattered patches of at- 
mospheric haze, and so on. Such environmental 
conditions should be carefully annotated on the imag- 
ery and the selection of training sites made based on 
the geographic stratification of these data. In such cas- 
es, it may be necessary to train the classifier over rela- 
tively short geographic distances. Each individual 
stratum may have to be classified separately. The final 
classification map of the entire region will then be a 
composite of the individual stratum classifications. 
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Relationship between Level of Detail Required and the Spatial Resolution 
of Representative Remote Sensing Systems for Vegetation Inventories 


Level I: Global 
AVHRR 
MODIS 
SPOT VEGETATION 
resolution: 250 m to 1.1 km 


Level II: Continental 
AVHRR 
MODIS 
Landsat Multispectral Scanner 
Landsat Thematic Mapper 
resolution: 80 m to 1.1 km 


Level III: Biome 
Landsat Multispectral Scanner 
Landsat Thematic Mapper Plus 
Synthetic Aperture Radar 
resolution: 30 m to 80 m 


Level IV: Region 
Landsat Thematic Mapper, ETM* 
SPOT 4, 5, 6 
High Altitude Aerial Photography 
Synthetic Aperture Radar 
resolution: 3m to 30m 


Level V: Plot 
Stereoscopic Aerial Photography 
IKONOS, GEOEye-1 
WorldView-1, -2 
resolution: 0.25 m to 3m 


Level VI: Jn situ Measurement 
Surface Measurements 
and Observations 


Generalized 
Vegetation 
Classification 


Upland Forest 


} Boreal Forest 


Lo Net =! 
Typical Study Area 


ye Ve | 


FIGURE 9-9 Relationship between the level of detail required and the spatial resolution of representative remote sensing 


systems for vegetation inventories. 


However, if environmental conditions are homoge- 
neous or can be held constant (e.g., through band ra- 
tioing or atmospheric correction), it may be possible to 
extend signatures vast distances in space, significantly 
reducing the training cost and effort. Additional re- 
search is required before the concept of geographic and 


temporal (through time) signature extension is fully 
understood (e.g., Jia et al., 2014). 


Once spatial and temporal signature extension factors 
have been considered, the analyst selects representative 
training sites for each class and collects the spectral 
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statistics for each pixel found within each training site. 
Each site is usually composed of many pixels. The gen- 
eral rule is that if training data are being extracted 
from n bands, then a minimum of >10n pixels of train- 
ing data are collected for each class. This is sufficient to 
compute the variance—covariance matrices required by 
some classification algorithms. 


There are a number of ways to collect the training site 
data, including: 


* collection of in situ information such as tree type, 
height, percent canopy closure, and diameter-at- 
breast-height (dbh) measurements, 


* on-screen selection of training data, and/or 
* on-screen seeding of training data. 


Ideally, each training site is visited in the field and its 
perimeter and/or centroid coordinates are measured di- 
rectly using a global positioning system (GPS). Be- 
cause the U.S. government has selective availability 
“off,” the differentially corrected GPS x, y coordinates 
should be within —1 m of their planimetric position. 
The GPS-derived x, y coordinates of a training site 
(e.g., a polygon enclosing a stand of oak) may then be 
input directly to the image processing system and used 
to extract training class (e.g., oak forest) statistics. 


The analyst may also view the image on the computer 
screen and select polygonal areas of interest (AOI) 
(e.g., a stand of oak forest). Most image processing 
systems use a “rubber band” tool that allows the ana- 
lyst to identify detailed AOIs. Conversely, the analyst 
may seed a specific location in the image using the cur- 
sor. The seed program begins at a single x, y location 
and evaluates neighboring pixel values in all bands of 
interest. Using criteria specified by the analyst, the 
seed algorithm expands outward like an amoeba as 
long as it finds pixels with spectral characteristics simi- 
lar to the original seed pixel. This is a very effective 
way of collecting homogeneous training information. 


Each pixel in each training site associated with a par- 
ticular class (c) is represented by a measurement vector, 
Xe 7 : 
B Vin 
B Vivo 


x, = |BY,; 


i73 (9.1) 


Prk 


where BV; ; ;, is the brightness value for the i,jth pixel in 
band k. The brightness values for each pixel in each 
band in each training class can then be analyzed statis- 


tically to yield a mean measurement vector, M,, for 
each class: 


Hey 
Hog 

M.= rae (9.2) 
[Pen] 


where KX, represents the mean value of the data ob- 
tained for class cin band k. The raw measurement vec- 
tor can also be analyzed to yield the covariance matrix 
for each class c: 


Cov.44 Cov.i7 a Covitn 


Vo = Vek = 


Cov .n4 Cov.o7 Ef Cov.nn (9.3) 


Coven] Covinn ahs Cov enn 


where Coy; ;1s the covariance of class c between bands 
k through /. For brevity, the notation for the covariance 
matrix for class c (i.¢e., V.,;) will be shortened to just 
V.. The same will be true for the covariance matrix of 
class d (1.¢., Vajg = Vg). 


The mean, standard deviation, variance, minimum val- 
ue, maximum value, variance—covariance matrix, and 
correlation matrix for the training statistics of five 
Charleston, SC, land-cover classes (residential, com- 
mercial, wetland, forest, and water) are listed in Table 
9-6. These represent fundamental information on the 
spectral characteristics of the five classes. 


Sometimes the manual selection of polygons results in 
the collection of training data with multiple modes in a 
training class histogram. This suggests that there are at 
least two types of land cover within the training area. 
This condition is not good when we are attempting to 
discriminate among individual classes. Therefore, it is a 
good practice to discard multimodal training data and 
retrain on specific parts of the polygon of interest until 
unimodal histograms are derived per class. 


Positive spatial autocorrelation exists among pixels that 
are contiguous or close together. This means that adja- 
cent pixels have a high probability of having similar 
brightness values. Training data collected from auto- 
correlated data tend to have reduced variance, which 
may be caused more by the way the sensor is collecting 
the data than by actual field conditions (e.g., most de- 
tectors dwell on an individual pixel for a very short 
time and may smear spectral information from one 
pixel to an adjacent pixel). The ideal situation is to col- 
lect training data within a region using every nth pixel 
or some other sampling criterion. The goal is to get 
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TABLE 9-6 Univariate and multivariate training statistics for the five land-cover classes using six bands of Landsat Thematic 


Mapper data of C 


harleston, SC. 


a. Statistics for Residential 
Band 1 Band 2 Band 3 Band 4 Band 5 Band 7 

Univariate Statistics 
Mean 70.6 28.8 29.8 36.7 55.7 28.2 
Std. dev. 6.90 3.96 5.65 4.53 10.72 6.70 
Variance 47.6 15.7 SY 20.6 114.9 44.9 
Minimum 5g) 22 19 26 32 16 
Maximum oil 41 45 52 84 48 
Variance—Covariance Matrix 

1 47.65 

2 24.76 15.70 

3 35.71 20.34 31.91 

4 12.45 8.27 12.01 20.56 

5 34.71 2307/9) 38.81 22.30 114.89 

7 30.46 18.70 30.86 12.99 60.63 44.92 
Correlation Matrix 

1 1.00 

2 0.91 1.00 

3 0.92 0.91 1.00 

4 0.40 0.46 0.47 1.00 

5 0.47 0.56 0.64 0.46 1.00 

7 0.66 0.70 0.82 0.43 0.84 1.00 
b. Statistics for Commercial 

Band 1 Band 2 Band 3 Band 4 Band 5 Band 7 

Univariate Statistics 
Mean 112.4 Se 63.5 54.8 77.4 45.6 
Std. dev. 5.77 4.55 S195 3.88 11.16 7.56 
Variance BS 20.7 15.6 150) 124.6 7/2 
Minimum 103 43 56 47 57 32 
Maximum 124 59 72 62 98 57 
Variance—Covariance Matrix 

1 33.29 

2 11.76 20.71 

3 19.13 11.42 15.61 
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b. Statistics for Commercial (Continued) 


Band 1 Band 2 Band 3 Band 4 Band 5 Band 7 

4 19.60 1207/ 14.26 15.03 

5 -16.62 15.84 ZnS) 0.94 124.63 

7 -4.58 17.15 6.94 5.76 68.81 57.16 
Correlation Matrix 

1 1.00 

2 0.45 1.00 

3 0.84 0.64 1.00 

4 0.88 0.72 0.93 1.00 

5 -0.26 0.31 0.05 0.02 1.00 

7 -0.10 0.50 0.23 0.20 0.82 1.00 
c. Statistics for Wetland 

Band 1 Band 2 Band 3 Band 4 Band 5 Band 7 

Univariate Statistics 
Mean 59.0 21.6 19.7 20.2 28.2 12.2 
Std. dev. 1.61 0.71 0.80 1.88 4.31 1.60 
Variance 2.6 ON 0.6 3:5 18.6 2.6 
Minimum 54 20 18 17 20 9) 
Maximum 63 25 21 25 35 16 
Variance—Covariance Matrix 

1 2.59 

2 0.14 0.50 

3 0.22 0.15 0.63 

4 -0.64 0.17 0.60 3.54 

5 -1.20 0.28 0.93 508} 18.61 

7 -0.32 0.17 0.40 1.72 4.53 2-15)5) 
Correlation Matrix 

1 1.00 

2 0.12 1.00 

3 0.17 0.26 1.00 

4 -0.21 0.12 0.40 1.00 

5 -0.17 0.09 0.27 OL.7/3} 1.00 

7 -0.13 0.15 0.32 0.57 0.66 1.00 
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d. Statistics for Forest 
Band 1 Band 2 Band 3 Band 4 Band 5 Band 7 

Univariate Statistics 
Mean 57.5 Zl a7/ 19.0 39.1 35.5 12.5 
Std. dev. 2.21 1.8) 1.40 5.11 6.41 2a 
Variance 4.9 le? Ne? 26.1 41.1 8.8 
Minimum 53) 20 Wy 25 22 8 
Maximum 63 28 24 48 54 22 
Variance—Covariance Matrix 

1 4.89 

2 1A 1.93 

3 2.05 1.54 1.95 

4 Bs) B95 4.06 26.08 

5 9.89 5.30 5.66 13.80 41.13 

7 4.63 2.34 aoe. BP22. 16.59 8.84 
Correlation Matrix 

1 1.00 

2 0.62 1.00 

3 0.66 0.80 1.00 

4 0.47 0.56 0.57 1.00 

5 0.70 0.59 0.63 0.42 1.00 

7 0.70 0.57 0.53 0.21 0.87 1.00 
e. Statistics for Water 

Band 1 Band 2 Band 3 Band 4 Band 5 Band 7 

Univariate Statistics 
Mean Gil E22 18.3 D8) 52 Delf 
Std. dev. 1.31 0.66 0.72 0.56 0.71 1.01 
Variance ‘l7/ 0.4 0.5 0.3 0.5 1.0 
Minimum 58 22 7 8 4 0 
Maximum 65 25 20 10 7 5 
Variance—Covariance Matrix 

1 1.72 

2 0.06 0.43 

3 0.12 0.19 0.51 

4 0.09 0.05 0.05 0.32 
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e. Statistics for Water (Continued) 


Band 1 Band 2 Band 3 Band 4 Band 5 Band 7 

5) -0.26 -0.05 -0.11 -0.07 0.51 

7 -0.21 -0.05 -0.03 -0.07 0.05 1.03 
Correlation Matrix 

1 1.00 

2 0.07 1.00 

3 0.13 0.40 1.00 

4 0.12 0.14 0.11 1.00 

5 -0.28 -0.10 -0.21 -0.17 1.00 

7 -0.16 -0.08 -0.04 -0.11 0.07 1.00 


non-autocorrelated training data. Unfortunately, most 
digital image processing systems do not provide this 
option in training data—collection modules. 


Selecting the Optimum Bands for 
Image Classification: Feature 
Selection 


Once the training statistics have been systematically 
collected from each band for each class of interest, a 
judgment must be made to determine the bands that 
are most effective in discriminating each class from all 
others. This process is commonly called feature selec- 
tion (Dalponte et al., 2009; Fuchs et al., 2009). The 
goal is to delete from the analysis the bands that pro- 
vide redundant spectral information. In this way the di- 
mensionality (i.c., the number of bands to be 
processed) in the dataset may be reduced. This process 
minimizes the cost of the digital image classification 
process (but should not affect the accuracy). Feature 
selection may involve both statistical and graphical 
analysis to determine the degree of between-class sepa- 
rability in the remote sensor training data (Lillesand et 
al., 2008). Using statistical methods, combinations of 
bands are normally ranked according to their potential 
ability to discriminate each class from all others using n 
bands at a time (Beauchemin and Fung, 2001; Lu and 
Weng, 2007; Fuchs et al., 2009). 


One might ask, “Why use graphical methods of feature 
selection if statistical techniques provide information 
sufficient to select the most appropriate bands for clas- 
sification?” The reason is simple. An analyst may base 
a decision solely on the statistic, yet never obtain a ba- 
sic understanding of the spectral nature of the data be- 
ing analyzed. In effect, without ever visualizing where 
the spectral measurements cluster in n-dimensional 
feature space, each new supervised classification finds 
the analyst beginning anew, relying totally on the ab- 


stract statistical analysis. Many practitioners of remote 
sensing are by necessity very graphically literate; that 
is, they can readily interpret maps and graphs. There- 
fore, a graphic display of the statistical data is useful 
and often necessary for a thorough analysis of multi- 
spectral training data and feature selection. Graphic 
feature selection methods have been developed for this 
purpose. 


Graphic Methods of Feature Selection 

Feature space plots in two dimensions depict the distri- 
bution of all the pixels in the scene using two bands at 
a time (Figure 9-10). Such plots are often used as a 
backdrop for the display of various graphic feature se- 
lection methods. A typical plot usually consists of a 
256K 256 matrix (0 to 255 in the x-axis and 0 to 255 in 
the y-axis), which is filled with values in the following 
manner. Let us suppose that the first pixel in the entire 
dataset has a brightness value of 50 in band | and a 
value of 30 in band 3. A value of 1 is placed at location 
50, 30 in the feature space plot matrix. If the next pixel 
in the dataset also has brightness values of 50 and 30 
in bands | and 3, the value of this cell in the feature 
space matrix is incremented by 1, becoming 2. This 
logic is applied to each pixel in the scene. The brighter 
the pixel is in the feature space plot display, the greater 
the number of pixels having the same values in the two 
bands of interest. 


Feature space plots provide insight into the actual in- 
formation content of the image and the degree of be- 
tween-band correlation. For example, in Figure 9-10a 
it is obvious that Thematic Mapper bands | (blue) and 
3 (red) are highly correlated and that atmospheric scat- 
tering in band 1 (blue) results in a significant shift of 
the brightness values down the x-axis. Conversely, 
plots of bands 2 (green) and 4 (near-infrared) and 
bands 3 (red) and 4 have a much greater distribution of 
pixels within the spectral space and some interesting 
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Two-dimensional Feature Space Plots 
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<E 9-10 Two-dimensional feature space plots of four pairs of Landsat TM data of Charleston, SC. a) TM bands 1 and 3, 


b) TM bands 2 and 4, c) TM bands 3 and 4, and d) TM bands 4 and 5. The brighter a particular pixel is in the display, the more 
pixels within the scene having that unique combination of band values. 


bright locations, which correspond with important 
land-cover types (Figures 9-10bc). Finally, the plot of 
bands 4 (near-infrared) and 5 (middle-infrared) shows 
exceptional dispersion throughout the spectral space 
and some very interesting bright locations (Figure 9- 
10d). For this reason, a spectral feature space plot of 
bands 4 and 5 will be used as a backdrop for the next 
graphical feature selection method. 


Cospectral parallelepiped or ellipse plots in two-dimen- 
sional feature space provide useful visual between-class 
separability information. They are produced using the 
mean, K,, and standard deviation, o ,,, of training 


class statistics for each class c and band k. For exam- 
ple, the original training statistics for five Charleston, 
SC, land-cover classes are portrayed in this manner 
and draped over the feature space plot of TM bands 4 
and 5 in Figure 9-11. The lower and upper limits of the 
two-dimensional parallelepipeds (rectangles) were ob- 
tained using the mean — lo of each band for each class. 
If only band 4 data were used to classify the scene, 
there would be confusion between classes 1 and 4, and 
if only band 5 data were used, there would be confu- 
sion between classes 3 and 4. However, when band 4 
and 5 data are used at the same time to classify the 
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Two-dimensional Feature Space Plot of Five Training Classes 
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FIGURE 9-11 Plot of the Charleston, SC, Landsat TM training statistics for five classes measured in bands 4 and 5 displayed 
as ‘cospectral parallelepipeds. The upper and lower limit of each parallelepiped is —10 . The parallelepipeds are superim- 


posed on a feature space plot of bands 4 and 5. 


scene there appears to be good between-class separa- 
bility among the five classes (at least at—1o). 


An evaluation of Figure 9-11 reveals that there are nu- 
merous water pixels in the scene found near the origin 
in bands 4 and 5. The water training class is located in 
this region. Similarly, the wetland training class is situ- 
ated within the bright wetland region of band 4 and 5 
spectral space. However, it appears that training data 
were not collected in the heart of the wetland region of 
spectral space. Such information is valuable because 
we may want to collect additional training data in the 
wetland region to see if we can capture more of the es- 


sence of the feature space. In fact, there may be two or 
more wetland classes residing in this portion of spec- 
tral space. Sophisticated digital image processing sys- 
tems allow the analyst to select training data directly 
from this type of display, which contains 1) the training 
class parallelepipeds, and 2) the feature space plot. The 
analyst uses the cursor to interactively select training 
locations (they may be polygonal areas, not just paral- 
lelepipeds) within the feature space. If desired, these 
feature space partitions can be used as the actual deci- 
sion logic during the classification phase of the project. 
This type of interactive feature space partitioning is 
very useful. 
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It is possible to display three bands of training data at 
once using trispectral parallelepipeds or ellipses in syn- 
thetic three-dimensional feature space (Figure 9-12). 
Jensen and Toll (1982) presented a method of display- 
ing parallelepipeds in synthetic three-dimensional 
space and of interactively varying the viewpoint azi- 
muth and elevation angles to enhance feature analysis 
and selection. Again, the mean, %.,, and standard de- 
viation, o .;, of training class statistics for each class c 
and band k were used to identify the lower and upper 
threshold values for each class and band. The analyst 
then selects a combination of three bands to portray 
because it is not possible to use all six bands at once in 
a three-dimensional display. Landsat TM bands 4, 5, 
and 7 are used in the following example; however, the 
method is applicable to any three-band subset. Each 
corner of a parallelepiped is identifiable by a unique set 
of x, y, z coordinates corresponding to either the lower 
or upper threshold value for the three bands under in- 
vestigation (Figure 9-12). 


The corners of the parallelepipeds may be viewed from 
a vantage point other than a simple frontal view of the 
x, y axes using three-dimensional coordinate transfor- 
mation equations. The feature space may be rotated 
about any of the axes, although rotation around the x- 
and y-axes normally provides a sufficient number of 
viewpoints. Rotation about the x-axis » radians and 
the y-axis 9 radians is implemented using the follow- 
ing equations (Hodgson and Plews, 1989): 


i 


P’ Pp’ 
[x, y,z,1] = |Bvx, BVy, BVz, 1] ° (9.4) 
1 0 0 O cos® 0 sinO 0 
0 cosh —-singd 0),| 0 1 0 0 
0 sind cosd 0} |—sin® 0 cosO 0 
0 0 0 1 0 0 0 1 


Negative signs of @ or 0 are used for counterclockwise 
rotation and positive signs for clockwise rotation. This 
transformation causes the original brightness value co- 
ordinates, P”, to be shifted anew and contain specified 
depth information as vector P’. Display devices are 
two-dimensional (e.g., plotter surfaces or LCD 
screens); ony the x and y elements of the transformed 
matrix P’ are used to draw the parallelepipeds. 


Manipulation of the transformed coordinates associat- 
ed with the Charleston, SC, training statistics is shown 
in Figure 9-13. All three of the bands (4, 5, and 7) are 
displayed in Figure 9-13a, except that the band 7 statis- 
tics are perpendicular (orthogonal) to the sheet of pa- 
per. By rotating the display 45 the contribution of 


Three-dimensional Parallelpiped 
xX 


Band 5 


Band 4 


FIGURE 9-12 Simple parallelepiped displayed in pseudo 
three-dimensional space. Each of the eight corners repre- 
sents a unique x, y, Z coordinate corresponding to a lower or 
upper threshold value of the training data. For example, the 
original coordinates of point 4 are associated with 1) the 
upper threshold value of band 4, 2) the lower threshold val- 
ue of band 5, and 3) the lower threshold value of band 7. 
The rotation matrix transformations cause the original coor- 
dinates to be rotated about the y-axis some 9 radians, and 
the x-axis some @ radians. 


band 7 becomes apparent (Figure 9-13b). This repre- 
sents a synthetic three-dimensional display of the par- 
allelepipeds. As the display is rotated another 45X to 
90K, band 7 data collapse onto what was the band 4 
axis (Figure 9-13c). The band 4 axis is now perpendicu- 
lar to the page, just as band 7 was originally. The band 
7, band 5 plot (Figure 9-13c) displays some overlap be- 
tween wetland (3) and forest (4). By systematically 
specifying various azimuth and elevation angles, it is 
possible to display the parallelepipeds for optimal visu- 
al examination. This allows the analyst to obtain in- 
sight as to the consistent location of the training data 
in three-dimensional feature space. 


In this example it is evident that just two bands, 4 and 
5, provide as good if not better separation than all 
three bands used together. However, this may not be 
the very best set of two bands to use. It might be useful 
to evaluate other two- or three-band combinations. In 
fact, a certain combination of perhaps four or five 
bands used all at one time might be superior. The only 
way to determine if this is true is to perform statistical 
feature selection. 
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Five Charleston, SC, Training 
Classes derived from Landsat 

Thematic Mapper Data 


FIGURE 9-13 Development of the three-dimensional parallelepipeds of the five Charleston, SC, training classes derived from 
Landsat Thematic Mapper data. Only bands 4, 5, and 7 are used in this investigation. The data are rotated about the y-axis, 
OX 45%, 90H At OX and 90K (parts [a] and [c], respectively) we are actually look ing at onl y two bands, analogous to the two- 
dimensional parallelepiped boxes shown in Figure 9-11. The third band lies perpendicular to the page we are viewing. 
Between such extremes, however, it is possible to obtain optimum viewing angles for visual analysis of training class statistics 
using three bands at once. Part (b) displays the five classes at a rotation of 45%, demonstrating that the classes are entirely 
separable using this three-band combination. However, it probably is not necessary to use all three bands since bands 4 and 
5 alone will discriminate satisfactorily between the five classes, as shown in part (a). There would be a substantial amount of 
overlap between classes 3 and 4 if only bands 5 and 7 were used. 


Statistical Methods of Feature Selection 

Statistical methods of feature selection are used to 
quantitatively select which subset of bands (also re- 
ferred to as features) provides the greatest degree of 
statistical separability between any two classes c and d. 
The basic problem of spectral pattern recognition is 
that given a spectral distribution of data in 1 bands of 
remotely sensed data, we must find a discrimination 
technique that will allow separation of the major land- 
cover categories with a minimum of error and a mini- 
mum number of bands. This problem is demonstrated 


diagrammatically using just one band and two classes 
in Figure 9-14. Generally, the more bands we analyze 
in a classification, the greater the cost and perhaps the 
greater the amount of redundant spectral information 
being used. When there is overlap, any decision rule 
that one could use to separate or distinguish between 
two classes must be concerned with two types of error 
(Figure 9-14): 


1. A pixel may be assigned to a class to which it does not 
belong (an error of commission). 
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2. A pixel is not assigned to its appropriate class (an 
error of omission). 


The goal is to select the optimum subset of bands and 
apply appropriate classification techniques to minimize 
both types of error in the classification process. If the 
training data for each class from each band are nor- 
mally distributed, as suggested in Figure 9-14, it is pos- 
sible to use either a transformed divergence or Jeffreys— 
Matusita distance equation to identify the optimum 
subset of bands to use in the classification procedure. 


Divergence Divergence was one of the first measures 
of statistical separability used in the machine process- 
ing of remote sensor data, and it is still widely used as 
a method of feature selection (e.g., Lu and Weng, 
2007; Dalponte et al., 2009; Exelis ENVI, 2013). It ad- 
dresses the basic problem of deciding what is the best 
g-band subset of n bands for use in the supervised clas- 
sification process. The number of combinations C of n 
bands taken g at a time is: 


At) = aenar 


Thus, if there are six TM bands and we are interested 
in the three best bands to use in the classification of the 
Charleston scene, this results in 20 combinations that 
must be evaluated: 


(9.5) 


6)_ __ 6!_—_s 720 
(3) ~ 31(6—3)!  6(6) 
= 20 combinations. 


If the best two-band combination was desired, it would 
be necessary to evaluate 15 possible combinations. 


Divergence is computed using the mean and covari- 
ance matrices of the class statistics collected in the 
training phase of the supervised classification. We will 
initiate the discussion by concerning ourselves with the 
statistical separability between just two classes, c and d. 
The degree of divergence or separability between c and 
d, Diver,y 18 computed according to the formula 
(Mausel et al., 1990): 


Diver, = serl(Ve- V. Vy -V,')] (9.6) 


+ serl( V,'+V,')(M,-M,)(M,-M,)'] 


where tr [ ] is the trace of a matrix (i.e., the sum of the 
diagonal elements), Vand V, are the covariance ma- 
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FIGURE 9-14 The basic problem in remote sensing pattern 
recognition classification is, given a spectral distribution of 
data in n bands (here just 1 band), to find an n-dimensional 
decision boundary that will allow the separation of the ma- 
jor classes (just 2 in this example) with a minimum of error 
and a minimum number of bands being evaluated. The dark 
areas of both distributions identify potential classification 
error. 


trices for the two classes under investigation, c and d, 
and M. and M, are the mean vectors for classes c and 
d (Konecny, 2014). It should be remembered that the 
sizes of the covariance matrices Vand V, are a func- 
tion of the number of bands used in the training pro- 
cess (i.e., if six bands were trained upon, then both V, 
and V,; would be matrices 6 X 6 in dimension). Diver- 
gence in this case would be used to identify the statisti- 
cal separability of the two training classes using six 
bands of training data. However, this is not the usual 
goal of applying divergence. What we actually want to 
know is the optimum subset of g bands. For example, if 
q = 3, what subset of three bands provides the best sep- 
aration between these two classes? Therefore, in our ex- 
ample, we would proceed to systematically apply the 
algorithm to the 20 three-band combinations, comput- 
ing the divergence for our two classes of interest and 
eventually identifying the subset of bands, perhaps 
bands 2, 3, and 6, that results in the largest divergence 
value. 


But what about the case where there are more than two 
classes? In this instance, the most common solution is 
to compute the average divergence, Diver,,,. This in- 
volves computing the average over all possible pairs of 
classes c and d, while holding the subset of bands q 
constant. Then, another subset of bands gq is selected 
for the m classes and analyzed. The subset of features 
(bands) having the maximum average divergence may 
be the superior set of bands to use in the classification 
algorithm. This can be expressed: 
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m-1 m 


¥ » Diver. 
c=1 d=ctl 


ave = 7 (9.7) 


Diver 


Using this, the band subset g with the highest average 
divergence would be selected as the most appropriate 
set of bands for classifying the m classes. 


Unfortunately, outlying easily separable classes will 
weight average divergence upward in a misleading 
fashion to the extent that suboptimal feature subsets 
might be indicated as best (Ozkan and Erbek, 2003; 
Richards, 2013). Therefore, it is necessary to compute 
transformed divergence, TDiver,,, expressed as: 


one . 0.8) 


TDiver,g = 2000] 1 - exp(— 
This statistic gives an exponentially decreasing weight 
to increasing distances between the classes. It also 
scales the divergence values to lie between 0 and 2,000. 
For example, Table 9-7 demonstrates which bands are 
most useful when taken 1, 2, 3, 4, or 5 at a time. There 
is no need to compute the divergence using all six 
bands since this represents the totality of the dataset. It 
is useful, however, to calculate divergence with individ- 
ual channels (g = 1), since a single channel might ade- 
quately discriminate among all classes of interest. 


A transformed divergence value of 2,000 suggests ex- 
cellent between-class separation. Above 1,900 provides 
good separation, while below 1,700 is poor. It can be 
seen that for the Charleston study, using any single 
band (Table 9-7a) would not produce results as accept- 
able as using bands 3 (red) and 4 (near-infrared) to- 
gether (Table 9-7b). Several three-band combinations 
should yield good between-class separation for all 
classes. Most of them understandably include TM 
bands 3 and 4. But why should we use three, four, five, 
or six bands in the classification when divergence sta- 
tistics suggest that very good between-class separation 
is possible using just two bands? We probably should 
not if the dimensionality of the dataset can be reduced 
by a factor of 3 (from 6 to 2) and classification results 
appear promising using just the two bands. 


Bhattacharyya Distance Other methods of feature 
selection are also based on determining the separabili- 
ty between two classes at a time. For example, the 
Bhattacharyya distance assumes that the two classes c 
and d are Gaussian and that the means M@, and M, and 
covariance matrices Vand V,, are available. The algo- 
rithm is (Ifarraguerri and Prairie, 2004; Lu and Weng, 
2007; Dalponte et al., 2009): 


1 Toe ta 
Bhat. = y(M.—M,) ( : ) (M.-M,) (9.9) 
Seg Via 
1 2 
+ slog, — 


UARIZ 


To select the best g features (i.e, combination of 
bands) from the original n bands in an m-class prob- 
lem, the Bhattacharyya distance is calculated between 
each m(m — 1)/2 pair of classes for each possible way of 
choosing g features from m dimensions. The best g fea- 
tures are those dimensions whose sum of the Bhat- 
tacharyya distance between the m(m-—1)/2 classes is 
highest (Konecny, 2014). 


Jeffreys-Matusita Distance A saturating transfor- 
mation applied to the Bhattacharyya distance measure 
yields the Jeffreys—Matusita distance (often referred to 
as the JM distance) computed using (Ferro and War- 
ner, 2002; Ifarraguerri and Prairie, 2004; Dalponte et 


al., 2009): 
—Bhat.., 
IM. = 201 —e Di 


The JM distance has a saturating behavior with in- 
creasing class separation like transformed divergence. 
However, it is not as computationally efficient as trans- 
formed divergence. 


(9.10) 


Ifarraguerri and Prairie (2004) developed a feature se- 
lection method that built on the Jeffreys—Matusita dis- 
tance measure. They counted the number of times that 
each band was selected as the best when used in combi- 
nation with other bands (1.e., 2 bands at a time, 3 bands 
at a time, etc.), during the band selection process. They 
plotted the band counts as two-dimensional histo- 
grams. The relative heights of the histogram bins are 
the relative frequencies with which each band is select- 
ed. The graphic provides a good visual indication of 
the desirability of each band in maximizing the dis- 
tance metric. 


Correlation Matrix Feature Selection The Corre- 
lation Matrix feature selection methodology is useful 
when trying to determine the optimum bands when 
creating vegetation and other types of indices. For ex- 
ample, Im et al. (2012) used Correlation Matrix feature 
selection to identify the optimum hyperspectral bands 
in the region from 400 to 2,500 nm to predict leaf-area- 
index (LAT) on hazardous waste sites in Utah and Ari- 
zona. The goal was to identify the optimum bands to 
use in the generic vegetation index: 
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TABLE 9-7 Divergence statistics for the five Charleston, SC, land-cover classes evaluated using 1, 2, 3, 4, and 5 Thematic 
Mapper band combinations at one time. 


Divergence (upper number) and Transformed Divergence (lower number) 


Class Combinations (1 = residential; 2 = commercial; 3 = wetland; 4 = forest; 5 = water) 


1845 


1 1 1 1 2 2 2 3 3 4 
Band Average 
Combinations Divergence 2 3 4 5 3 4 5 4 5 5 
a. One Band at a Time 
1 45 36 23 38 600 356 803 1 3) 7 
1583 1993 sez, 1889 1982 2000 2000 2000 198 651 1145 
2 34 67 15 54 1036 286 1090 1 5 5 
1588 1970 2000 1786 1998 2000 2000 2000 246 988 890 
3 54 107 39 160 1591 576 2071 1 3 1 
1525 1998 2000 1985 2000 2000 2000 2000 286 642 339) 
4 19 47 0 1238 209 ils 3357 60 210 1466 
1748 1809 1994 70 2000 2000 1603 2000 1999 2000 2000 
5 4 26 7 2645 Ti 29 5300 2 556 961 
1636 779 1920 1194 2000 2000 1947 2000 523 2000 2000 
H 6 61 18 345 238 74 940 1 63 56 
1707 1061 OX) 1795 2000 2000 2000 2000 213 999 1998 
b. Two Bands at a Time 
12 Sil 92 26 85 1460 410 1752 ez 8 10 
1709 1997 2000 OS 2000 2000 2000 2000 463 1256 1457 
i 2 56 125 40 182 1888 589 2564 2 7 lif 
1709 1998 2000 1987 2000 2000 2000 2000 418 1196 1490 
1 4 55 100 32 253 941 446 S199. 66 219 1525 
1996 1998 2000 1962 2000 2000 2000 2000 1999 2000 2000 
1 54 WA 28 3072 778 497 7838 6 585 1038 
1896 1998 2000 1939 2000 2000 2000 2000 1029 2000 2000 
i 7 52 107 28 426 944 421 2065 3} 63 76 
1852 1997 2000 11939 2000 2000 2000 2000 586 Oy 2000 
23 By/ 140 42 170 2099 593 2345 2 13 2 
1749 1998 2000 1990 2000 2000 2000 2000 524 1599 1382 
2 4 35 103 28 1256 1136 356 3985 65 228 (529 
1992 1976 2000 1941 2000 2000 2000 2000 1999 2000 2000 
2 3 35 86 20 2195) 1068 328 6932 4 560 979 
1856 1976 2000 1826 2000 2000 2000 2000 760 2000 2000 
2a 37 111 24 423 1148 292 2192 Z 69 66 
1829 1980 2000 1902 2000 2000 2000 2000 405 2000 1999 
3 4 101 124 61 1321 1606 905 4837 80 210 1487 
2000 2000 2000 S99) 2000 2000 2000 2000 2000 2000 2000 
3 5 59 114 45 3206 1609 740 9142 5 597 1024 
1895 1999 2000 1992 2000 2000 2000 2000 964 2000 2000 
3} 63 131 41 525 1610 606 3122 2 65 59 


1999 2000 1989 2000 2000 2000 2000 469 1999 1999 
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TABLE 9-7 Divergence statistics for the five Charleston, SC, land-cover classes evaluated using 1, 2, 3, 4, and 5 Thematic 
Mapper band combinations at one time. (Continued) 


Divergence (upper number) and Transformed Divergence (lower number) 


Class Combinations (1 = residential; 2 = commercial; 3 = wetland; 4 = forest; 5 = water) 


1 tl 1 1 2 2 2 3 3 4 
Band Average 
Combinations Divergence 2 3 4 5 3 4 5 4 5 5 
4 5 2A 52 11 4616 231 37 10376 98 889 2902 
1930 1851 1997 1468 2000 2000 1981 2000 2000 2000 2000 
4 7 20 76 21 1742 309 79 4740 86 285 1599 
1970 1844 2000 1857 2000 2000 2000 2000 2000 2000 2000 
Ss) If 6 62 24 2870 246 97 5956 5 598 989 
195 1074 1999. 1900 2000 2000 2000 2000 978 2000 2000 
c. Three Bands at a Time 
i 23 59 154 44 191 2340 613 2821 3 16 17 
1815 1999 2000 1992 2000 2000 2000 2000 643 1745 1774 
1 2 a 95 142 40 1266 1662 675 4381 68 236 1573 
1999 2000 2000 1986 2000 2000 2000 2000 2000 2000 2000 
1 2 8 58 118 32 3201 1564 604 9281 7 589 1045 
1909 1999 2000 1964 2000 2000 2000 2000 1129 2000 2000 
1 2 7 ay/ 146 30 493 1653 494 3176 4 69 80 
1868 1998 2000 1953 2000 2000 2000 2000 732 2000 2000 
eh 117 150 64 1329 1905 985 5120 86 219 1534 
2000 2000 2000 1999 2000 2000 2000 2000 2000 2000 2000 
12 60 137 51 3569 1902 Yer) || 1172241 7 622 1088 
1920 1999 2000 1997 2000 2000 2000 2000 1202 2000 2000 
1 3 7 63 157 45 580 oss 669 3879 4 66 79 
1872 IO 2000 IOVS) 2000 2000 2000 2000 731 IO 2000 
(45 82 105 36 4923 978 635] 12361 104 906 2955 
1998 2000 2000 1979 2000 2000 2000 2000 2000 2000 2000 
1 4 7 82 129 37 WT 1055 610 5452 98 288 1669 
1998 2000 2000 1980 2000 2000 2000 2000 2000 2000 2000 
i Ss 7 56 109 SY/ 3405 956 508 8948 8 627 1077 
1924 1998 2000 1982 2000 2000 2000 2000 1261 2000 2000 
23 4 117 156 63 esi 2119 956 4971 81 229 1530 
2000 2000 2000 1999 2000 2000 2000 2000 2000 2000 2000 
23D 62 147 47 3221 2120 749 9480 6 605 1034 
1908 1999 2000 1994 2000 2000 2000 2000 1082 2000 2000 
A 3 7 66 160 46 541 2113 617 3480 3 74 69 
1865 1999 2000 1994 2000 2000 2000 2000 661 2000 2000 
24°55 38 108 31 4674 1158 385] 11402 103 896 2946 
1994 1984 2000 1956 2000 2000 2000 2000 2000 2000 2000 
24 7 40 125 34 WA nga 367 Soild 90 300 1668 
1996 1986 2000 1970 2000 2000 2000 2000 2000 2000 2000 
2 i 38 113 33 3050 Sy 365 Vil H 594 1006 
1906 1982 2000 1968 2000 2000 2000 2000 Hills 2000 2000 
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TABLE 9-7 Divergence statistics for the five Charleston, SC, land-cover classes evaluated using 1, 2, 3, 4, and 5 Thematic 
Mapper band combinations at one time. (Continued) 


Divergence (upper number) and Transformed Divergence (lower number) 


Class Combinations (1 = residential; 2 = commercial; 3 = wetland; 4 = forest; 5 = water) 


1 1 1 1 2 2 2 3 3 4 
Band Average 

Combinations Divergence 2 3 4 5 3 4 5 4 5 5 
3.4 5 106 129 65 5031 1622 1037} 13505 120 914 2935 
2000 2000 2000 1999 2000 2000 2000 2000 2000 2000 2000 
3) Uh 7/ 4 144 63 1841 1644 955 6309 102 285 1626 
2000 2000 2000 1999 2000 2000 2000 2000 2000 2000 2000 
3 S 7 66 134 63 3453 1648 823 9900 8 631 1054 
1927 1999 2000 1999 2000 2000 2000 2000 1268 2000 2000 
45 7 22 83 26 5003 362 114] 11477 105 944 2994 
1979 1870 2000 1923 2000 2000 2000 2000 2000 2000 2000 


d. Four Bands at a Time 


2000 


1 28 4 167 177 65 (S39 2361 1151 5259 87 238 1575 
2000 2000 2000 1999 2000 2000 2000 2000 2000 2000 2000 
1 235 63 165 54 3582 2355 GAS || les 8 630 1095 
1929 1999 2000 1998 2000 2000 2000 2000 1294 2000 2000 
1 23 7 67 182 49 S25) 2369 683 4222 5 v5 87 
1888 2000 2000 1996 2000 2000 2000 2000 885 2000 2000 
1 2h 115 147 46 4971 1696 901 13287 108 Als} 2987 
1999 2000 2000 1994 2000 2000 2000 2000 2000 2000 2000 
1 2 2b ye 110 165 45 1801 1731 868 6161 %6 303 WZZ5 
1999 2000 2000 1073} 2000 2000 2000 2000 2000 2000 2000 
1 25 7 61 148 41 3564 1665 614} 10579 9 633 1085 
1932 1999 2000 1989 2000 2000 2000 2000 1331 2000 2000 
1) 3 455 133 156 74 5293 esa 1283} 15187 127 928 2976 
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
13 4 7 134 172 69 1863 1955 1184 6814 110 289 1682 
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
1 3 > 7 66 159 66 3919 1954 901 12411 10 665 1129 
1940 2000 2000 2000 2000 2000 2000 2000 1397 2000 2000 
1¢ 5 7 88 135 42 5422 1105 659} 13950 lz 970 3068 
1999 2000 2000 1990 2000 2000 2000 2000 2000 2000 2000 
2 Sea) 122 161 67 5040 2133 1093} 13663 121 933 2981 
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
ya oi Uh 7 132 173 65 1848 2143 1023 6509 103 302 1670 
2000 2000 2000 1999 2000 2000 2000 2000 2000 2000 2000 
A 3 B I 69 163 68 3476 2144 837} 10308 > 639 1062 
1937 2000 2000 2000 2000 2000 2000 2000 1370 2000 2000 
2A Se, 41 (3a 38 5079 1229 397| 12641 110 O53 3037 
nor 1987 2000 1983 2000 2000 2000 2000 2000 2000 2000 
st) oy 7 112 148 74 5436 1665 1066} 14688 125 971 3030 


2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
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TABLE 9-7 Divergence statistics for the five Charleston, SC, land-cover classes evaluated using 1, 2, 3, 4, and 5 Thematic 


Mapper band combinations at one time. (Continued) 


Divergence (upper number) and Transformed Divergence (lower number) 
Class Combinations (1 = residential; 2 = commercial; 3 = wetland; 4 = forest; 5 = water) 
1 1 1 1 2 2 2 3 3 4 
Band Average 
Combinations Divergence 2 3 4 5 3 4 5 4 5 5 
e. Five Bands at a Time 
12345 176 183 U5 5302 2384 1422} 15334 128 947 3019 
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
12347 176 196 va 1871 2393 1316 7015 lit 305 1726 
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
1A35 7 70 184 72 3940 2386 O19) |e Z798 i 673 dis5 
1948 2000 2000 2000 2000 2000 2000 2000 1479 2000 2000 
12457 ly 171 50 5487 1770 920} 15021 115 977 3101 
2000 2000 2000 1996 2000 2000 2000 2000 2000 2000 2000 
(SA Sie7, 138 176 80 5803 1979 1294] 16829 132 994 3089 
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
23457 134 177 Wi 5443 2161 1130} 14893 126 987 3072 
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 
By = By 
VI2 = 2,500 
B 2 +B 1 Correlation Matrix 


where B, and B, are bands in the region from 400 to 
2,500 nm and B, is always smaller in value than Bp. 
The coefficient of determination (R?) was computed 
between the VI2 under investigation and in situ LAI 
measurements for each of the possible 15,750 possible 
VI2 band combinations. The darker the shade of blue 
in Figure 9-15, the higher the correlation between the 
VI2 and LAI. The darker the shade of red, the lower 
the correlation between the VI2 and LAI. Mahlein et 
al. (2013) used a related feature extraction algorithm 
called RELIEF-F to identify the optimum bands for 
creating Sugarbeet rust and powdery mildew detection 
indices from hyperspectral data. 


Fuchs et al. (2009) used stepwise linear regression 
modeling to select which original QuickBird and re- 
mote sensing—derived variables to use in their predic- 
tion of above ground carbon. The variables tested 
included original bands, Kauth-Thomas Tasseled Cap, 
Principal Components Analysis (PCA), numerous veg- 
etation indices and texture measures. 


Principal Components Analysis (PCA) Feature 

Selection As discussed in Chapter 8, principal com- 
ponents analysis (PCA) can be applied to a multispec- 
tral dataset to identify the principal components and 
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FIGURE 9-15 An example of the use of a Correlation Ma- 
trix feature (band) selection. The goal was to identify the 
optimum bands from 400 to 2,500 nm for use in the generic 
vegetation index VI2 = (B2—-B,) / (B>+B,) regressed against 
in situ LAl measurements. The higher the coefficient of de- 
termination (R2) for each of the 15,750 possible VI2s re- 
gressed with in situ LAl measurements, the darker the 
shade of blue. The darker the shade of red, the lower the 
correlation between a VI2 and LAI (adapted from Im et al., 
2012). 


the amount of variance explained by each component. 
The newly-created PCA images that account for the 
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FIGURE 9-16 Points a and b are pixels in the image to be classified. Pixel a has a brightness value of 40 in band 4 and 40 in 
band _ 5. Pixel b has a brightness value of 10 in band 4 and 40 in band 5. The boxes represent the parallelepiped decision rule 
associated with a—16 classification. The vectors (arrows) represent the distance from a and b to the mean of all classes in a 
minimum distance to means classification algorithm. Refer to Tables 9-8 and 9-9 for the results of classifying points a and b 


using both classification techniques. 


most variance can then be used when collecting train- 
ing data instead of the original remote sensor data. In 
this situation, PCA in effect performs feature selection 
and reduces the dimensionality of the dataset that is 
used for training and classification (Rocchini et al., 
2007). 


Select the Appropriate Classification 
Algorithm 


Various supervised classification algorithms can be 
used to assign an unknown pixel to one of m possible 
classes (Jensen et al., 2009). The choice of a particular 
classifier or decision rule depends on the nature of the 
input data and the desired output and is very impor- 
tant. Parametric classification algorithms assume that 
the observed measurement vectors X. obtained for 
each class in each spectral band during the training 
phase of the supervised classification are Gaussian; 
that is, they are normally distributed (Schowengerdt, 
2007). Conversely, nonparametric classification algo- 
rithms make no such assumption (Lu and Weng, 
2007). 


Several widely adopted nonparametric classification 
algorithms include: 


* one-dimensional density slicing (e.g., using a single 
band of imagery; please refer to Chapter 8), 


¢ parallepiped, 
* minimum distance, 
* nearest-neighbor, and 


¢ neural network and expert system analysis (Chapter 
10). 


The most widely adopted parametric classification al- 
gorithms is: 


¢ the maximum likelihood algorithm. 


It is instructive to review the logic of several classifica- 
tion algorithms in this and subsequent chapters. 


Parallelepiped Classification Algorithm 

This digital image classification decision rule is based 
on simple Boolean “and/or” logic. Training data in n 
spectral bands are used to perform the classification. 
Along with other algorithms, it is used in sophisticated 
object-based image analysis (OBIA) programs such as 
eCognition (Trimble, 2014). Brightness values from 
each pixel of the multispectral imagery are used to pro- 
duce an n-dimensional mean vector, M. = (x, Ker, Ke3, 
..., &,) with X., being the mean value of the training 
data obtained for class c in band k out of m possible 
classes, as previously defined. o ,, is the standard devi- 
ation of the training data class c of band k out of m 
possible classes. In this discussion we will evaluate all 
five Charleston classes using just bands 4 and 5 of the 
training data. 
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TABLE 9-8 Example of parallelepiped classification logic for pixels a and b in Figure 9-16. 


Does pixel a (40, 40) Does pixel b (10, 40) 
satisfy criteria satisfy criteria 
for this class for this class 
in this band? in this band? 
Class Lower Threshold, L,, Upper Threshold, H., La<a< Hey La<b< He, 
1.Residential 
Band 4 36.7 — 4.53 = 31.27 36.7 + 4.53 = 41.23 Yes No 
Band 5 55.7 — 10.72 = 44.98 55.7 + 10.72 = 66.42 No No 
2.Commercial 
Band 4 54.8 — 3.88 = 50.92 54.8 + 3.88 = 58.68 No No 
Band 5 77.4-—11.16 = 66.24 77.4 + 11.16 = 88.56 No No 
3.Wetland 
Band 4 20.2 — 1.88 = 18.32 20.2 + 1.88 = 22.08 No No 
Band 5 28.2 — 4.31 = 23.89 28.2 + 4.31 = 32.51 No No 
4.Forest 
Band 4 39.1 -5.11 = 33.99 39.14 5.11 = 44.21 Yes No 
Band 5 35.5 — 6.41 = 29.09 35.5 + 6.41 = 41.91 Yes, assign pixel to class No 
4, forest. STOP. 
5.Water 
Band 4 9.3 -—0.56 = 8.74 9.3 + 0.56 = 9.86 —_— No 
Band 5 5.2-0.71 = 4.49 5.2 + 0.71 = 5.91 — No, assign pixel to un- 
classified category. 
STOP. 


Using a one-standard-deviation threshold (as shown in 
Figure 9-16), a parallelepiped algorithm decides BV jx 
is in class c if, and only if: 


Wer — F ck S BV jg S Bex + © ex (9.11) 
where 
c=1,2,3,...,m, number of classes, and 
k=1,2,3,...,n, number of bands. 


Therefore, if the low and high decision boundaries are 
defined as 


Lex a Bek — Ok (9. 12) 
and 
Ax = Mek + 6 ck (2.13) 
the parallelepiped algorithm becomes 
Lg a BV 7 = Dg (9.14) 


These decision boundaries form an n-dimensional par- 
allelepiped in feature space. If the pixel value lies above 
the low threshold and below the high threshold for all 
n bands evaluated, it is assigned to that class (see point 
a in Figure 9-16). When an unknown pixel does not 
satisfy any of the Boolean logic criteria (point b in Fig- 
ure 9-16), it is assigned to an “unclassified” category. 
Although it is only possible to analyze visually up to 
three dimensions, as described in the section on com- 


puter graphic feature analysis, it is straightforward to 
create an n-dimensional parallelepiped for classifica- 
tion purposes. 


We will review how unknown pixels a and b are as- 
signed to the forest and unclassified categories in Fig- 
ure 9-16. The computations are summarized in Table 
9-8. First, the standard deviation is subtracted and 
added to the mean of each class and for each band to 
identify the lower (Z,,;) and upper (/7,;) edges of the 
parallelepiped. In this case only two bands are used, 4 
and 5, resulting in a two-dimensional box. This could 
be extended to n dimensions or bands. With the lower 
and upper thresholds for each box identified, it is pos- 
sible to determine if the brightness value of an input 
pixel in each band, k, satisfies the criteria of any of the 
five parallelepipeds. For example, pixel a has a value of 
40 in both bands 4 and 5. It satisfies the band 4 criteria 
of class 1 (1.¢., 31.27 < 40 < 41.23), but it does not satis- 
fy the band 5 criteria. Therefore, the process continues 
by evaluating the parallelepiped criteria of classes 2 
and 3, which are also not satisfied. However, when the 
brightness values of a are compared with class 4 
thresholds, we find it satisfies the criteria for band 4 
(i.e., 33.99 < 40 < 44.21) and band 5 (29.09 < 40 < 
41.91). Thus, the pixel is assigned to class 4, forest. 


This same logic is applied to classify unknown pixel 0. 
Unfortunately, its brightness values of 10 in band 4 and 
40 in band 5 never fall within the thresholds of any of 
the parallelepipeds. Consequently, it is assigned to an 
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unclassified category. Increasing the size of the thresh- 
olds to —2 or 3 standard deviations would increase the 
size of the parallelepipeds. This might result in point b 
being assigned to one of the classes. However, this same 
action might also introduce a significant amount of 
overlap among many of the parallelepipeds resulting in 
classification error. Perhaps point b really belongs to a 
class that was not trained upon (e.g., dredge spoil). 


The parallelepiped algorithm is a computationally effi- 
cient method of classifying remote sensor data. Unfor- 
tunately, because some parallelepipeds overlap, it is 
possible that an unknown candidate pixel might satisfy 
the criteria of more than one class. In such cases it is 
usually assigned to the first class for which it meets all 
criteria. A more elegant solution is to take this pixel 
that can be assigned to more than one class and use a 
minimum distance to means decision rule to assign it to 
just one class. 


Minimum Distance to Means Classification 
Algorithm 


The minimum distance to means decision rule is compu- 
tationally simple and commonly used. When used 
properly it can result in classification accuracy compa- 
rable to other more computationally intensive algo- 
rithms such as the maximum likelihood algorithm. 
Like the parallelepiped algorithm, it requires that the 
user provide the mean vectors for each class in each 
band X_, from the training data. To perform a mini- 
mum distance classification, a program must calculate 
the distance to each mean vector KX, from each un- 
known pixel (BV;,) (Lo and Yeung, 2007). It is possi- 
ble to calculate this distance using Euclidean distance 
based on the Pythagorean theorem or “round the 
block” distance measures (Figure 9-17). In this discus- 
sion we demonstrate the method of minimum distance 
classification using Euclidean distance measurements 
applied to the two unknown points (a and b) shown in 
Figure 9-17. 


The computation of the Euclidean distance from point 
a (40, 40) to the mean of class 1 (36.7, 55.7) measured 
in bands 4 and 5 relies on the equation: 


v} 2 
Dist = [ere Hew + (BV ;;)~ He) (9.15) 
where X,, and K., represent the mean vectors for class c 


measured in bands k and /. In our example this would 
be: 


; 2 2 
Dist is ane t= [@%4-14,0 +(BViis—by 5) (9.16) 


The distance from point a to the mean of class 2 in 
these same two bands would be: 


0 b; a) 


Round the block distance 


/ D 2 
D5 = = (aii D 4p = = ai-Pal 


FIGURE 9-17 The distance used in a minimum distance to 
means classification algorithm can take two forms: the Eu- 
clidean distance based on the Pythagorean theorem and 
the “round the block” distance. The Euclidean distance is 
more computationally intensive. 


Euclidean distance 


2 2 
Dist 5 dines = [¥ 4-H9,0 + (BV i5—y 5) (9.17) 


Notice that the subscript that stands for class c is incre- 
mented from | to 2. By calculating the Euclidean dis- 
tance from point a to the mean of all five classes, it is 
possible to determine which distance is shortest. Table 
9-9 is a list of the mathematics associated with the 
computation of distances for the five land-cover class- 
es. It reveals that pixel a should be assigned to class 4 
(forest) because it obtained the minimum distance of 
4.59.The same logic can be applied to evaluate the un- 
known pixel 4. It is assigned to class 3 (wetland) be- 
cause it obtained the minimum distance of 15.75. It 
should be obvious that any unknown pixel will defi- 
nitely be assigned to one of the five training classes us- 
ing this algorithm. There will be no unclassified pixels. 


Most minimum distance classification algorithms let 
the analyst specify a distance or threshold from the 
class means beyond which a pixel will not be assigned 
to a category even though it is nearest to the mean of 
that category. For example, if a threshold of 10.0 were 
specified, point a would still be classified as class 4 (for- 
est) because it had a minimum distance of 4.59, which 
was below the threshold. Conversely, point b would 
not be assigned to class 3 (wetland) because its mini- 
mum distance of 15.75 was greater than the 10.0 
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TABLE 9-9 Example of minimum distance to means classification logic for pixels a and b in Figure 9-16. 


Distance from Pixel a (40, 40) 


Distance from Pixel b (10, 40) 


Class to the Mean of Each Class to the Mean of Each Class 
1. Residential (40 — 36.7)2 + (40 — 55.7)? = 16.04 (10 — 36.7)2 + (40 — 55.7)? = 30.97 


2. Commercial 


(40 — 54.8)2 + (40 —77.4)2 = 40.22 


(10 — 54.8)2 + (40 —77.4)2 = 58.35 


3. Wetland N(40 — 20.2)? + (40 — 28.2)? = 23.04 N(10 — 20.2)? + (40 — 28.2)? = 15.75 
Assign pixel b to this class; it has the minimum 
distance. 

4. Forest (40 — 39.1)2 + (40 —35.5)2 = 4.59 (10 —39.1)2 + (40 — 35.5)2 = 29.45 

Assign pixel a to this class; it has the minimum 
distance. 
5. Water (40 — 9.3)? + (40 — 5.2)? = 46.4 (10 — 9.3)? + (40 — 5.2)? = 34.8 


threshold. Instead, point b would be assigned to an un- 
classified category. 


When more than two bands are evaluated in a classifi- 
cation, it is possible to extend the logic of computing 
the distance between just two points in n space using 
the equation (Duda et al., 2001): 


(9.18) 


Figure 9-17 demonstrates diagrammatically how this 
algorithm is used. 


Hodgson (1988) identified six additional Euclidean- 
based minimum distance algorithms that decrease 
computation time by exploiting two areas: 1) the com- 
putation of the distance estimate from the unclassified 
pixel to each candidate class, and 2) the criteria for 
eliminating classes from the search process, thus avoid- 
ing unnecessary distance computations. Algorithms 
implementing these improvements were tested using up 
to 2, 4, and 6 bands of TM data and 5, 20, 50, and 100 
classes. All algorithms were more efficient than the tra- 
ditional Euclidean minimum distance algorithm. 


A traditional minimum distance to means classifica- 
tion algorithm was run on the Charleston, SC, Landsat 
Thematic Mapper dataset using the training data pre- 
viously described. The results are displayed as a color- 
coded thematic map in Figure 9-18a. The total number 
of pixels in each class are summarized in Table 9-10. 
Error associated with the classification is discussed in 
Chapter 13. 


Nearest-Neighbor Classifiers 

The parallelepiped classifier made use of the training 
class mean and standard deviation statistics for each 
band. The minimum distance classifier required the 
mean for each training class for each band. It is also 
possible to classify an unknown pixel measurement 
vector into m classes using just the training data 
brightness values in each band (i.e., the mean and stan- 
dard deviation statistics are not used) and nearest- 
neighbor distance measurements (Duda et al., 2001; 
Fuchs et al., 2009). The most common nonparametric 
nearest-neighbor classifiers are: 


* nearest-neighbor, 
¢ k-nearest-neighbor, and 
¢ k-nearest-neighbor distance-weighted. 


The simple nearest-neighbor classifier computes the 
Euclidean distance from the pixel to be classified to the 
nearest training data pixel in n-dimensional feature 
space and assigns it to that class (Schowengerdt, 2007; 
Myint et al., 2011). 


The nearest-neighbor classifier makes no assumptions 
about the data being normally distributed. Given a 
pixel to be classified (z) and a training pixel (f), the 
equation to compute the nearest neighbor distance be- 
tween them is (Jensen et al., 2009): 


b 
2 
din a > (BV,,— BV) 


i=1 
where BV; is the brightness value for unlabeled pixel z 
in band 7, and BV ,; is the brightness value of a training 
pixel ¢ in band 7. During the pixel assignment, the pixel 
is given the label of the nearest neighbor that generates 


(9.19) 
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Minimum Distance to Means 
Supervised Classification of Charleston, SC, 
Using Landsat Thematic Mapper Data 
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Unsupervised Classification 


Class Legend Class 

1. Water 11. Commercial 2 
2. Forest 12. Commercial 2 
3. Forest 13. Commercial 2 
4. Wetland 14. Residential 

5. Wetland 15. Commercial 2 
6. Residential 16. Commercial 2 
7. Residential 17. Residential 

8. Park/golf 18. Residential 

9. Residential 19. Commercial 1 
10. Commercial 1 20. Commercial 2 


FIGURE 9-18 a) The results of applying a minimum distance to means classification algorithm to Landsat Thematic Mapper 
bands 4 and 5 data of Charleston, SC. Table 9-10 summarizes the number of pixels in each class.b) The results of performing 
an unsupervised classification of the Charleston, SC, Landsat Thematic Mapper imagery using bands 3, 4, and 5. Twenty 

spectral clusters were extracted and relabeled as information classes according to the criteria shown in Figure 9-29 and Table 
9-11. Note that the unsupervised classification extracted a Park/golf class and was able to identify two classes of commercial 


land cover. Original imagery courtesy of NASA. 


the smallest value of d,,,, among all the training pixels. 
For example, in the example in Figure 9-19, the near- 
est-neighbor logic would assign the unknown pixel 
measurement vector to the commercial class. 


The k-nearest-neighbor classifier searches away from 
the pixel to be classified in all directions until it en- 
counters k user-specified training pixels (e.g., k = 5). It 
then assigns the pixel to the class with the majority of 
pixels encountered. For example, the pixel under inves- 
tigation in Figure 9-19 would be assigned to the resi- 
dential class if a k-nearest-neighbor classifier were 


applied because three of the five training class pixels 
encountered within the circle are residential. 


The k-nearest-neighbor distance-weighted classifier uses 
the same k-nearest neighbors but it weights them ac- 
cording to the distance-weighting logic discussed in 
Chapter 7. The pixel under investigation is assigned to 
the training class pixel with the highest total weight. 
The pixel under investigation in Figure 9-19 would be 
assigned to the commercial land-use class once again if 
the k-nearest-neighbor distance-weighted algorithm 
was applied. 
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FIGURE 9-19 Hypothetical example of nearest-neighbor classification. Simple nearest-neighbor classification locates the 
near est training class pixel in n-dimensional feature space from the unknown pixel measurement vector under consideration 
and assigns the unknown pixel the class of the training data. The k-nearest-neighbor classifier locates the nearest k (e.g., 5) 
training pixels (no matter wha t their class)in feature space and assigns the pixel under investigation to the majority class. The 
k-nearest-neighbor distance-weighted algorithm measures the distance to the same k training class pixels and weights each 
of them according to their distance to the pixel under investigation. The pixel is assigned the class with the highest total 


weight. 


TABLE 9-10 Total number of pixels classified into each of 
the five Charleston, SC, land-cover classes shown in Figure 
9-18. 


Class Total Number of Pixels 
1. Residential 14,398 
2. Commercial 4,088 
3. Wetland 10,772 
4. Forest 11,673 
5. Water 20,509 


Nearest-neighbor classifiers can be relatively slow be- 
cause of the number of distance calculations required 
between the unknown pixel measurement vector and 
all of the training pixels in the various bands (Hardin, 
1994). But, nearest-neighbor classifiers can yield very 
useful results if the training data are well separated in 
n-dimensional feature space. Otherwise, it will proba- 
bly be necessary to use a different algorithm (e.g., min- 
imum distance to means, maximum likelihood, neural 
network). 


Maximum Likelihood Classification Algorithm 
The aforementioned classifiers were based primarily on 
identifying decision boundaries in feature space based 
on training class multispectral distance measurements. 
The nax imum |ilelihood decision rule is based on 
probability. It assigns each pixel having pattern mea- 
surements or features X to the class 7 whose units are 
most probable or likely to have given rise to feature 
vector XY (Lo and Yeung, 2007; Dalponte et al., 2009). 
In other words, the probability of a pixel belonging to 
each of a predefined set of m classes is calculated, and 
the pixel is then assigned to the class for which the 
probability is the highest. The maximum likelihood de- 
cision rule continues to be one of the most widely used 
supervised classification algorithms (e.g., Campbell 
and Wynne, 2011; Myint et al., 2011). 


The maximum likelihood classification procedure as- 
sumes that the training data statistics for each class in 
each band are normally distributed (i.e., Gaussian). 
Training data with bi- or n-modal histograms in a sin- 
gle band are not ideal. In such cases the individual 
modes probably represent unique classes that should 
be trained upon individually and labeled as separate 
training classes. This should then produce unimodal, 
Gaussian training class statistics that fulfill the normal 
distribution requirement. 
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a. Histogram (data frequency distribution) of 
forest training data in a single band k. 
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b. Data distribution approximated by a 
normal probability density function. 


FIGURE 9-20 a) Hypothetical histogram of the forest training data in band k. b) Probability density function computed for 
the hypothetical training data using Equation 9.20. This function can be used to approximate the frequency of occurrence for 


every value on the x-axis. 


But how do we obtain the probability information we 
will need from the remote sensing training data we 
have collected? The answer lies first in the computation 
of probability density functions. We will demonstrate 
using a single class of training data based on a single 
band of imagery. For example, consider the hypotheti- 
cal histogram (data frequency distribution) of forest 
training data obtained in band & shown in Figure 9- 
20a. We could choose to store the values contained in 
this histogram in the computer, but a more elegant so- 
lution is to approximate the distribution by a normal 
probability density function (curve), as shown super- 
imposed on the histogram in Figure 9-20b. The esti- 
mated probability function for class w; (e.g., forest) is 
computed using the equation: 


2 
alae | (9.20) 


P(x|w;) = tT lo 2 


Qn) 6, 


where exp [ ] is e (the base of the natural logarithms) 
raised to the computed power, x is one of the bright- 
ness values on the x-axis, p.is the estimated mean of all 
the values in the forest training class, and oy is the es- 
timated variance of all the measurements in this class. 
Therefore, we need to store only the mean and variance 
of each training class (e.g., forest) to compute the prob- 
ability function associated with any of the individual 
brightness values in it. 


But what if our training data consist of multiple bands 
of remote sensor data for the classes of interest? Then, 
we compute an n-dimensional multivariate normal 
density function using (Swain and Davis, 1978): 


p(X|w,) = exp[-3X-M,)'V; XM) (9.21) 


1 
n 
Qn)" [V4 


where lV; is the determinant of the covariance matrix, 
v, _1s the inverse of the covariance matrix, and (X — 
M, a is the transpose of the vector (XY — M;). The mean 
vectors (V/;) and covariance matrix (V;) for each class 
are estimated from the training data. For example, 
consider Figure 9-21 where the bivariate probability 
density functions of six hypothetical classes are ar- 
rayed in red and near-infrared feature space. It is bivar- 
late because two bands are used. Note how the 
probability density function values appear to be nor- 
mally distributed (1.e., bell-shaped). The vertical axis is 
associated with the probability of an unknown pixel 
measurement vector XY being a member of one of the 
classes. In other words, if an unknown measurement 
vector has brightness values such that it lies within the 
wetland region, it has a high probability of being wet- 
land land cover. 


If we assume that there are m classes, then p(X/w,) is 
the probability density function associated with the un- 
known measurement vector X, given that XY is from a 
pattern in class w; (Swain and Davis, 1978). In this case 
the maximum likelihood decision rule becomes (Rich- 
ards, 2013): 


Decide X € wW; if, and only if, 
P(X|w;) - p(w,) 2 p(X|w,) - Ow) (9.22) 
for all and j out of 1, 2,... m possible classes. 


Therefore, to classify a pixel in the multispectral re- 
mote sensing dataset with an unknown measurement 
vector XY, a maximum likelihood decision rule com- 
putes the product P(X|w,) -Ppw,) for each class and as- 
signs the pattern to the class having the largest 
product. This assumes that we have some useful infor- 
mation about the prior probabilities of each class i (i.e., 


P(W))). 
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Probability Density Functions Derived from Multispectral Training Data 


Probability Density Function Value, p(x Iw) 


FIGURE 9-21 Example of normally distributed probability density functions associated with six hypothetical land-cover class- 
es using two bands of multispectral data (red and near-infrared). The vertical axis is the probability of an unknown pixel mea- 


surement vector X being a member of one of the classes. 


Maximum Likelihood Classification Without Prior 

Probability Information In practice, we rarely have 
prior information about whether one class (e.g., forest) 
is expected to occur more frequently in a scene than 
any other class (e.g., 60% of the scene should be forest). 
This is called class a priori probability information (1.e., 
p(w;)). Therefore, most applications of the maximum 
likelihood decision rule assume that each class has an 
equal probability of occurring in the landscape. This 
makes it possible to remove the prior probability term 
[p(w,)] in Equation 9.22 and develop a simplified deci- 
sion rule that can be applied to the unknown measure- 
ment vector X for each pixel in the scene (Dalponte et 
al., 2009): 


Decide unknown measurement vector _X is in class 7 if, 
and only if, 


DizD; (9.23) 
for all i and j out of 1, 2, ... 7 possible classes 
and 
1 1 T_-l 
p; = ~5le,|V|- 3(X-M,)) V, (x-M))| (9.24) 


where M; is the mean measurement vector for class i 
and V; is the covariance matrix of class i for bands k 
through /. Therefore, to assign the measurement vector 
X of an unknown pixel to a class, the maximum likeli- 
hood decision rule computes the value p; for each class. 
Then it assigns the pixel to the class that has the largest 
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FIGURE 9-22 Hypothetical example of how the maximum likelihood decision rule functions when two training class probabil- 
ity density functions (forest and agriculture) measured in two bands overlap. The unknown measurement vector X associated 
with a single pixel in a two-band dataset would be assigned to forest because the probability density of its measurement 


vector X is greater for forest than for agriculture. 


(or maximum) value. This assumes that we have no 
useful information about the prior probabilities of 
each of the land-cover classes, that is, every class has 
an equal probability of occurring throughout the real- 
world landscape. 


Now let us consider the computations required. In the 
first pass, p; is computed with V; and M, being the co- 
variance matrix and mean vectors for class 1. Next, p> 
is computed with V> and M> being the covariance ma- 
trix and mean vectors for class 2. This continues for all 
m classes. The pixel or measurement vector YX is as- 
signed to the class that produces the largest or maxi- 
mum p;. The measurement vector X used in each step 
of the calculation consists of m elements (the number of 
bands being analyzed). For example, if six Landsat TM 
bands (i.e, no thermal band) were being analyzed, 
each unknown pixel would have a measurement vector 
X of: 


BV... 1 
lJ, 
Bye 
BYR). 


BY; 1,4 


(9.25) 


BY; 1,5 


barred 
But what happens when the probability density func- 
tions of two or more training classes overlap in feature 
space? For example, consider two hypothetical normal- 
ly distributed probability density functions associated 
with forest and agriculture training data measured in 
bands 1| and 2 (Figure 9-22). In this case, pixel Y would 
be assigned to forest because the probability density of 
unknown measurement vector X is greater for forest 
than for agriculture. 
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Maximum Likelihood Classification with Prior 
Probability Information Equation 9.23 assumes 
that each class has an equal probability of occurring in 
the terrain. Common sense reminds us that in most re- 
mote sensing applications there is a high probability of 
encountering some classes more often than others. For 
example, in the Charleston scene the probability of en- 
countering various land covers 1s: 


p(w) = p(residential) = 0.2, 
P(w2) = p(commercial) = 0.1, 
P(w3) = p(wetland) = 0.25, 
p(w) = p(forest) = 0.1, and 
P(ws) = p(water) = 0.35. 


Thus, we would expect more pixels to be classified as 
water simply because it is more prevalent in the image. 
If we have such information, it is possible to include 
this valuable prior knowledge in the classification deci- 
sion. We can do this by weighting each class 7 by its ap- 
propriate a priori probability, p(w;). The equation then 
becomes: 


Decide unknown measurement vector X is in class 7 if, 
and only if, 


P;POw,) =p; p(w.) (9.26) 


for all i and / out of 1, 2, ... 7 possible classes 
and 
PD; P(w,) = log, p(w,)— sloz,|V | 
-[5x- My Vv," (x- M)| 


This Bayesian decision rule is identical to the maxi- 
mum likelihood decision rule except that it does not as- 
sume that each class has equal probabilities. A priori 
probabilities have been used successfully as a way of in- 
corporating the effects of relief and other terrain char- 
acteristics in improving classification accuracy. The 
maximum likelihood and Bayes classifications require 
many more computations per pixel than either the par- 
allelepiped or minimum distance classification algo- 
rithms. These algorithms do not always produce 
superior classification results. 


We Unsupervised Classification 


Unsupervised classification (also commonly referred to 
as clustering) 1s an effective method of partitioning re- 
mote sensor image data in multispectral feature space 
and extracting land-cover information (Huang, 2002; 
Exelis ENVI, 2013). Compared to supervised classifi- 
cation, unsupervised classification normally requires 
only a minimal amount of initial input from the ana- 
lyst. This is because clustering does not normally re- 
quire training data. 


Unsupervised classification is the process whereby nu- 
merical operations are performed that search for natu- 
ral groupings of the spectral properties of pixels, as 
examined in multispectral feature space. The clustering 
process results in a classification map consisting of m 
spectral classes (Lo and Yeung, 2007). The analyst 
then attempts a posteriori (after the fact) to assign or 
transform the spectral classes into thematic information 
classes of interest (e.g., forest, agriculture, urban). This 
may not be easy. Some spectral clusters may be mean- 
ingless because they represent mixed classes of Earth 
surface materials. It takes careful thinking by the ana- 
lyst to unravel such mysteries. The analyst must under- 
stand the spectral characteristics of the terrain well 
enough to be able to label certain clusters as specific in- 
formation classes. 


Hundreds of clustering algorithms have been devel- 
oped (Duda et al., 2001; Schowengerdt, 2007). Two ex- 
amples of conceptually simple but not necessarily 
efficient clustering algorithms will be used to demon- 
strate the fundamental logic of unsupervised classifica- 
tion of remote sensor data. 


Unsupervised Classification Using 
the Chain Method 


The first clustering algorithm that will be discussed op- 
erates in a two-pass mode (i.¢., it passes through the 
multispectral dataset two times). In the first pass, the 
program reads through the dataset and sequentially 
builds clusters (groups of points in spectral space). A 
mean vector is then associated with each cluster. In the 
second pass, a minimum distance to means classifica- 
tion algorithm similar to the one previously described 
is applied to the whole dataset on a pixel-by-pixel basis 
whereby each pixel is assigned to one of the mean vec- 
tors created in pass |. The first pass, therefore, auto- 
matically creates the cluster signatures to be used by 
the minimum distance to means classifier. 
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FIGURE 9-23 Original values of pixels 1, 2, and 3 as mea- 
sured in bands 4 and 5 of the hypothetical remotely sensed 
data. 


Pass 1: Cluster Building 
During the first pass, the analyst is required to supply 
four types of information: 


1. R, a radius distance in spectral space used to 
determine when a new cluster should be formed 
(e.g., when raw remote sensor data are used, it 
might be set at 15 brightness value units), 


2. C, a spectral space distance parameter used when 
merging clusters (e.g., 30 units) when N is reached, 


3. N, the number of pixels to be evaluated between 
each major merging of the clusters (e.g., 2,000 
pixels), and 


4. Cnax the maximum number of clusters to be 
identified by the algorithm (e.g., 20 clusters). 


These can be set to default values if no initial human 
interaction is desired. 


Starting at the origin of the multispectral dataset (i.e., 
line 1, column 1), pixels are evaluated sequentially 
from left to right as if in a chain. After one line is pro- 
cessed, the next line of data is considered. We will ana- 
lyze the clustering of only the first three pixels in a 
hypothetical image and label them pixels 1, 2, and 3. 
The pixels have brightness values in just two bands, 4 
and 5. Their spatial relationships in two-dimensional 
feature space are shown in Figure 9-23. 


First, we let the brightness values associated with pixel 
1 in the image represent the mean data vector of cluster 
1 (.e., M, = {10, 10}). It is important to remember it is 
an n-dimensional mean data vector with n being the 
number of bands used in the unsupervised classifica- 
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FIGURE 9-24 The distance (D) in two-dimensional spectral 
space between pixel 1 (cluster 1) and pixel 2 (cluster 2) in 
the first iteration is computed and tested against the value 
of R, the minimum acceptable radius. In this case, D does 
not exceed R; therefore, we merge clusters 1 and 2, as 
shown in Figure 9-25. 


tion. In our example, just two bands are being used, so 
n = 2. Because we have not identified all 20 spectral 
clusters (Ciyyax) yet, pixel 2 will be considered as the 
mean data vector of cluster 2 (i.e, M> = {20, 20}). If 
the spectral distance D between cluster 2 and cluster | 
is greater than R, then cluster 2 will remain cluster 2. 
However, if the spectral distance D is less than R, then 
the mean data vector of cluster 1 becomes the average 
of the first and second pixel brightness values and the 
weight (or count) of cluster 1 becomes 2 (Figure 9-24). 
In our example, the distance D between cluster | (actu- 
ally pixel 1) and pixel 2 is 14.14. Because the radius R 
was initially set at 15.0, pixel 2 does not satisfy the cri- 
teria for being cluster 2 because its distance from clus- 
ter 1 is <15. Therefore, the mean data vectors of cluster 
1 and pixel 2 are averaged, yielding the new location of 
cluster 1 at M, = {15, 15}, as shown in Figure 9-25. 
The spectral distance D is computed using the Pythag- 
orean theorem, as discussed previously. 


Next, pixel 3 is considered as the mean data vector of 
cluster 2 (1.e., M@> = {30, 20}). The distance from pixel 
3 to the revised location of cluster 1, My = {15, 15}, is 
15.81 (Figure 9-25). Because it is >15, the mean data 
vector of pixel 3 becomes the mean data vector associ- 
ated with cluster 2. 


This cluster accumulation continues until the number 
of pixels evaluated is greater than N. At that point, the 
program stops evaluating individual pixels and looks 
closely at the nature of the clusters obtained thus far. It 
calculates the distance between each cluster and every 
other cluster. Any two clusters separated by a spectral 
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FIGURE 9-25 Pixels 1 and 2 now represent cluster 1. Note 
that the location of cluster 1 has migrated from (10, 10) to 
(15, 15), after the first iteration. Now, pixel 3 distance (D) is 
computed to see if it is greater than the minimum thresh- 
old, R. It is, so pixel location 3 becomes cluster 2. This pro- 
cess continues until all 20 clusters are identified. Then the 
20 clusters are evaluated using a distance measure, C (not 
shown in the figure), to merge the clusters that are closest 
to one another. 


distance less than C are merged. Such a new cluster 
mean vector is the weighted average of the two original 
clusters, and the weight is the sum of the two individual 
weights. This proceeds until there are no clusters with a 
separation distance less than C. Then the next pixel is 
considered. This process continues to iterate until the 
entire multispectral dataset is examined. 


Schowengerdt (2007) suggests that virtually all the 
commonly used clustering algorithms use iterative cal- 
culations to find an optimum set of decision boundar- 
ies for the dataset. It should be noted that some 
clustering algorithms allow the analyst to initially seed 
the mean vector for several of the important classes. 
The seed data are usually obtained in a supervised 
fashion, as discussed previously. Others allow the ana- 
lyst to use a priori information to direct the clustering 
process. 


Some programs do not evaluate every line and every 
column of the data when computing the mean vectors 
for the clusters. Instead, they may sample every ith row 
and jth column to identify the C,,,, clusters. If comput- 
er resources are abundant, then every pixel may be sam- 
pled. If resources are scarce, then acceptable results 
may usually be obtained by sampling the data. Obvi- 
ously, a great number of computations are performed 
during this initial pass through the dataset. 
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FIGURE 9-26 How clusters migrate during the several iter- 
ations of a clustering algorithm. The final ending point rep- 
resents the mean vector that would be used in phase 2 of 
the clustering process when the minimum distance classifi- 
cation is performed. 


A hypothetical diagram showing the cluster migration 
for our two-band dataset is shown in Figure 9-26. No- 
tice that as more points are added to a cluster, the 
mean shifts less dramatically since the new computed 
mean is weighted by the number of pixels currently in a 
cluster. The ending point is the spectral location of the 
final mean vector that is used as a signature in the min- 
imum distance classifier applied in pass 2. 


Pass 2: Assignment of Pixels to One of the 
Crax Clusters Using Minimum Distance 
Classification Logic 


The final cluster mean data vectors are used in a mini- 
mum distance to means classification algorithm to 
classify all the pixels in the image into one of the Cyyax 
clusters. The analyst usually produces a cospectral plot 
display to document where the clusters reside in three- 
dimensional feature space. It is then necessary to evalu- 
ate the location of the clusters in the image, label them 
if possible, and see if any should be combined. It is 
usually necessary to combine some clusters. This is 
where an intimate knowledge of the terrain is critical. 


An unsupervised classification of the Charleston, SC, 
Landsat TM scene is displayed in Figure 9-18b. It was 
created using TM bands 2, 3, and 4. The analyst stipu- 
lated that a total of 20 clusters (Cy,,,) be extracted 
from the data. The mean data vectors for each of the fi- 
nal 20 clusters are summarized in Table 9-11. These 
mean vectors represented the data used in the mini- 
mum distance classification of every pixel in the scene 
into one of the 20 cluster categories. 
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TABLE 9-11 Results of clustering on Thematic Mapper Bands 2, 3, and 4 of the Charleston, SC, Landsat TM scene. 


Mean Vector 
Percent of Class Color 
Cluster Scene Band 2 Band 3 Band 4 Description Assignment 
1 24.15 23.14 18.75 O15) Water Dark blue 
2 7.14 21.89 18.99 44.85 Forest 1 Dark green 
4 7.00 Cale) 1O.72 38.17 Forest 2 Dark green 
4 11.61 21.79 19.87 19.46 Wetland 1 Bright green 
5 5.83 22.16 20.51 23.90 Wetland 2 Bright green 
6 2.18 28.35 28.48 40.67 Residential 1 Bright yellow 
7 3.34 36.30 25.58 35.00 Residential 2 Bright yellow 
8 2.60 29.44 29.87 49.49 Parks, golf Gray 
9 1.72 32.69 34.70 41.38 Residential 3 Yellow 
10 1.85 26.92 26.31 28.18 Commercial 1 Dark red 
11 1.27 36.62 39.83 41.76 Commercial 2 Bright red 
12 0.53 44.20 49.68 46.28 Commercial 3 Bright red 
1s) 1.03 33.00 34.55 28.21 Commercial 4 Red 
14 1.92 30.42 31.36 36.81 Residential 4 Yellow 
15 1.00 40.55 44.30 89109 Commercial 5 Bright red 
16 2.13 35.84 38.80 35.09 Commercial 6 Red 
17 4.83 25.54 24.14 43.25 Residential 5 Bright yellow 
18 1.86 31.03 32.57 32.62 Residential 6 Yellow 
19 3.26 22.36 20.22 S21 Commercial 7 Dark red 
20 0.02 34.00 43.00 48.00 Commercial 8 Bright red 


Cospectral plots of the mean data vectors for each of 
the 20 clusters using bands 2 and 3 and bands 3 and 4 
are displayed in Figures 9-27 and 9-28, respectively. 
The 20 clusters lie on a diagonal extending from the or- 
igin in the band 2 versus band 3 plot. Unfortunately, 
the water cluster was located in the same spectral space 
as forest and wetland when viewed using just bands 2 
and 3 (Figure 9-27). Therefore, this scatterplot was not 
used to /abel or assign the clusters to information class- 
es. Conversely, a cospectral plot of bands 3 and 4 mean 
data vectors was relatively easy to interpret as shown in 
Figure 9-28. 


Cluster labeling is usually performed by interactively 
displaying all the pixels assigned to an individual clus- 
ter on the screen with a color composite of the study 


area in the background. In this manner it is possible to 
identify the location and spatial association among 
clusters. This interactive visual analysis in conjunction 
with the information provided in the cospectral plot al- 
lows the analyst to group the clusters into information 
classes, as shown in Figure 9-29 and Table 9-11. It is in- 
structive to review some of the logic that resulted in the 
final unsupervised classification in Figure 9-18b. 


Cluster 1 occupied a distinct region of spectral space 
(Figure 9-29). It was not difficult to assign it to the in- 
formation class water. Clusters 2 and 3 had high reflec- 
tance in the near-infrared (band 4) with low reflectance 
in the red (band 3) due to chlorophyll absorption. 
These two clusters were assigned to the forest class and 
color-coded dark green (refer to Table 9-11). Clusters 4 
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FIGURE 9-27 The mean vectors of the 20 clusters dis- 
played in Figure 9-18b are shown here using only bands 2 
and 3. The mean vector values are summarized in Table 9- 
11. Notice the substantial amount of overlap among clus- 
ters 1 through 5 and 19. 


and 5 were situated alone in spectral space between the 
forest (2 and 3) and water (1) and were comprised of a 
mixture of moist soil and abundant vegetation. There- 
fore, it was not difficult to assign both these clusters to 
a wetland class. They were given different color codes 
to demonstrate that, indeed, two separate classes of 
wetland were identified. 


Six clusters were associated with residential housing. 
These clusters were situated between the forest and 
commercial clusters. This is not unusual since residen- 
tial housing is composed of a mixture of vegetated and 
nonvegetated (asphalt and concrete) surfaces, especial- 
ly at TM spatial resolutions of 30 X 30 meters. Based 
on where they were located in feature space, the six 
clusters were collapsed into just two: bright yellow (6, 
7, 17) and yellow (9, 14, 18). 


Eight clusters were associated with commercial land 
use. Four of the clusters (11, 12, 15, 20) reflected high 
amounts of both red and near-infrared energy as com- 
mercial land composed of concrete and bare soil often 
does. Two other clusters (13 and 16) were associated 
with commercial strip areas, particularly the down- 
town areas. Finally, there were two clusters (10 and 19) 
that were definitely commercial but that had a substan- 
tial amount of associated vegetation. They were found 
mainly along major thoroughfares in the residential ar- 
eas where vegetation is more plentiful. These three sub- 
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FIGURE 9-28 The mean vectors of the 20 clusters displayed 
in Figure 9-18b are shown here using only bands 3 and 4 da- 
ta. The mean vector values are summarized in Table 9-11. 


groups of commercial land use were assigned bright 
red, red, and dark red, respectively (Table 9-11). Clus- 
ter 8 did not fall nicely into any group. It experienced 
very high near-infrared reflectance and chlorophyll ab- 
sorption often associated with very well kept lawns or 
parks. In fact, this is precisely what it was labeled, 
“parks and golf.” 


The 20 clusters and their color assignments are shown 
in Table 9-11 and in Figure 9-18b. There is more infor- 
mation present in this unsupervised classification than 
in the supervised classification. Except for water, there 
are at least two classes in each land-use category that 
could be identified successfully using the unsupervised 
technique. The supervised classification simply did not 
sample many of these classes during the initial training 
process. 


Unsupervised Classification Using 
the ISODATA Method 


Another widely used clustering algorithm is the Itera- 
tive Self-Organizing Data Analysis Technique (ISO- 
DATA). ISODATA represents a comprehensive set of 
heuristic (rule of thumb) procedures that have been in- 
corporated into an iterative classification algorithm 
(ERDAS, 2013; Pasher and King, 2010; Rich et al., 
2010). Many of the steps used in the algorithm are a re- 
sult of experience gained through experimentation. The 
ISODATA algorithm is a modification of the k-means 
clustering algorithm, which includes a) merging clus- 
ters if their separation distance in multispectral feature 
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FI@ RE 9-29 Grouping (labeling) of the original 20 spec- 
tral clusters into information classes used to create the the- 
matic map shown in Figure 9-18b. The labeling was 
performed by analyzing the mean vector locations in bands 
3 and 4. 


space is below a user-specified threshold, and b) rules 
for splitting a single cluster into two clusters (Memars- 
adeghi et al., 2007; Schowengerdt, 2007; Exelis ENVI, 
2013). 


ISODATA is self-organizing because it requires rela- 
tively little human input. A sophisticated ISODATA 
algorithm normally requires the analyst to specify the 
following criteria: 


Cmax: the maximum number of clusters to be identi- 
fied by the algorithm (e.g., 20 clusters). However, it 
is not uncommon for fewer to be found in the final 
classification map after splitting and merging take 
place. 


¢ 7: the maximum percentage of pixels whose class 
values are allowed to be unchanged between itera- 
tions. When this number is reached, the ISODATA 
algorithm terminates. Some datasets may never 
reach the desired percentage unchanged. If this hap- 
pens, it is necessary to interrupt processing and edit 
the parameter. 


¢ M: the maximum number of times ISODATA is to 
classify pixels and recalculate cluster mean vectors. 
The ISODATA algorithm terminates when this 
number is reached. 

© Minimum members in a cluster (%): If a cluster con- 
tains less than the minimum percentage of members, 
it is deleted and the members are assigned to an 


alternative cluster. This also affects whether a class 
is going to be split (see maximum standard devia- 
tion). The default minimum percentage of members 
is often set to 0.01. 


¢ Maximum standard deviation (6 max): When the 
standard deviation for a cluster exceeds the specified 
maximum standard deviation and the number of 
members in the class is greater than twice the speci- 
fied minimum members in a class, the cluster is split 
into two clusters. The mean vectors for the two new 
clusters are the old class centers -lo. Maximum 
standard deviation values between 4.5 and 7 are 
very typical. 

¢ Split separation value: If this value is changed from 
0.0, it takes the place of the standard deviation in 
determining the locations of the new mean vectors 
plus and minus the split separation value. 


¢ Minimum distance between cluster means (C).: Clus- 
ters with a weighted distance less than this value are 
merged. A default of 3.0 is often used. 


Memarsadeghi et al. (2007) summarize the characteris- 
tics of the fundamental ISODATA clustering algo- 
rithm and then provide a fast implementation. They 
provide the C** code for the fundamental algorithm as 
well as the fast implementation at http://www.cs.umd 
.edu/~mount/Projects/ISODATA/. 


ISODATA Initial Arbitrary Cluster Allocation 


ISODATA is iterative because it makes a large number 
of passes through the remote sensing dataset until 
specified results are obtained, instead of just two pass- 
es. Also, ISODATA does not allocate its initial mean 
vectors based on the analysis of pixels in the first line 
of data the way the two-pass algorithm does. Rather, 
an initial arbitrary assignment of all Cy, clusters 
takes place along an n-dimensional vector that runs be- 
tween very specific points in feature space. The region 
in feature space is defined using the mean, X,, and 
standard deviation, o ;, of each band in the analysis. A 
hypothetical two-dimensional example using bands 3 
and 4 is presented in Figure 9-30a, in which five mean 
vectors are distributed along the vector beginning at 
location K3 — 0 3, My4— o4and ending at MH, + o3, Ky t+ 
o 4. This method of automatically seeding the original 
Cmax vectors makes sure that the first few lines of data 
do not bias the creation of clusters. Note that the two- 
dimensional parallelepiped (box) does not capture all 
the possible band 3 and 4 brightness value combina- 
tions present in the scene. The location of the initial 
Cmax Mean vectors (Figure 9-30a) should move about 
somewhat to partition the feature space better. This 
takes place in the first and subsequent iterations. 
Huang (2002) developed an efficient version of ISO- 
DATA that provides improved automatic location of 
the initial mean vectors (i.e., seed clusters). 
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FIGURE 9-30 a) ISODATA initial distribution of five hypothetical mean vectors using —16 standard deviations in both bands 
as beginning and ending points. b) In the first iteration, each candidate pixel is compared to each cluster mean and assigned 
to the cluster whose mean is closest in Euclidean distance. c) During the second iteration, a new mean is calculated for each 
cluster based on the actual spectral locations of the pixels assigned to each cluster, instead of the initial arbitrary calculation. 
This involves analysis of several parameters to merge or split clusters. After the new cluster mean vectors are selected, every 
pixel in the scene is assigned to one of the new clusters. d) This split-merge—assign process continues until there is only a 
small amount of change in class assignment between iterations (the T threshold is reached) or the maximum number of itera- 


tions is reached (M). 


ISODATA First Iteration 


With the initial C,,,, mean vectors in place, a pass is 
made through the database beginning in the upper left 
corner of the matrix. Each candidate pixel is compared 
to each cluster mean and assigned to the cluster whose 
mean is closest in Euclidean distance (Figure 9-30b). 


This pass creates an actual classification map consist- 
ing of C,y4x classes. It should be noted that some image 
processing systems process data line by line, and others 
process the data in a block or tiled data structure. The 
way that ISODATA is instructed to process the data 
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FIGURE 9-31 a) Distribution of 20 ISODATA mean vectors after just one iteration using Landsat TM band 3 and 4 data of 
Charleston, SC. Notice that the initial mean vectors are distributed along a diagonal in two-dimensional feature space accord- 
ing to the —2o standard deviation logic discussed. b) Distribution of 20 ISODATA mean vectors after 20 iterations. The bulk 
of the important feature space (the yellow background) is partitioned rather well after just 20 iterations. 


(e.g., line by line or block by block) will have an impact 
on the creation of the mean vectors. 


ISODATA Second to Mth Iteration 


After the first iteration, a new mean for each cluster is 
calculated based on the actual spectral locations of the 
pixels assigned to each cluster, instead of on the initial 
arbitrary calculation. This involves analysis of the fol- 
lowing parameters: minimum members in a cluster 
(%), maximum standard deviation (6 »,4x), split sepa- 
ration, and minimum distance between cluster means 
(C). Then the entire process is repeated with each can- 
didate pixel once again compared to the new cluster 
means and assigned to the nearest cluster mean (Fig- 
ure 9-30c). Sometimes individual pixels do not change 
cluster assignment. This iterative process continues 
(Figure 9-30d) until there is 1) little change in class as- 
signment between iterations (i.e., the 7 threshold is 
reached), or 2) the maximum number of iterations is 
reached (M). The final file is a matrix with C,,,, clus- 
ters in it, which must be labeled and recoded to become 
useful land-cover information. The fact that the initial 
mean vectors are situated throughout the heart of the 
existing data is superior to initiating clusters based on 
finding them in the first line of the data. 


The iterative ISODATA algorithm is relatively slow, 
and image analysts are notoriously impatient. Analysts 


must allow the ISODATA algorithm to iterate enough 
times to generate meaningful mean vectors. 


ISODATA Example 1 An ISODATA classification 
was performed using the Charleston, SC, Landsat TM 
band 3 and 4 data. The locations of the clusters (mean 
—2o) after one iteration are shown in Figure 9-3la. 
The clusters are superimposed on the distribution of all 
brightness values found in TM bands 3 and 4. The lo- 
cation of the final mean vectors after 20 iterations is 
shown in Figure 9-31b. The ISODATA algorithm has 
partitioned the feature space effectively. Requesting 
more clusters (e.g., 100) and allowing more iterations 
(e.g., 500) would partition the feature space even bet- 
ter. A classification map example is not provided be- 
cause it would not be dramatically different from the 
results of the two-pass clustering algorithm since so 
few clusters were requested. 


ISODATA Example 2. An ISODATA algorithm was 
applied to two bands of HyMap hyperspectral data 
(red and near-infrared) of an area near North Inlet, 
SC. The goal was to extract six land-cover classes from 
the data: water, wetland, roof/asphalt, forest, bare soil, 
and golf fairway. Only 10 clusters were extracted using 
ISODATA for demonstration purposes. The locations 
of the 10 clusters (actually mean vectors) after 1-5 and 
10 iterations are displayed in Figure 9-32. The red and 
near-infrared spectral characteristics of the final 10 
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FIGURE 9-32 a-f) Distribution of 10 ISODATA mean vectors after 1-5 and 10 iterations using just two bands of HyMap hy- 
perspectral data of an area near North Inlet, SC. The 10 mean vectors after 10 iterations were used to produce the thematic 
map shown in Figure 9-33. g) Typically, more than 20 clusters are requested when using ISODATA. This graphic shows the dis- 
tribution of the two-dimensional feature space for 20 clusters after 20 iterations. 
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Information C asses Derived from an ISODATA Unsupervised ClassiXcation 
Using 10 Iterations and 10 Mean Vectors of an Area Near North Inlet, SC 
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b. Final location of 10 ISODATA mean vectors after 10 iterations. 


FIGURE 9-33 ISODATA classification of HyMap remote sensor data of an area near North Inlet, SC. a) Color composite im- 
age. b) Location of 10 mean vectors in red and near-infrared feature space after 10 iterations. c) Classification map derived by 


labeling the 10 spectral classes as information classes. 


mean vectors after 10 iterations are found in Figure 9- 
32f. Typically, 20 or more clusters are requested. 
Therefore, Figure 9-32g shows where 20 clusters reside 
after 20 iterations. 


The 10 mean vectors shown in Figure 9-32f were used 
in the final pass of the ISODATA algorithm to create a 


thematic map consisting of 10 spectral classes. These 
spectral classes were evaluated and labeled as the 10 in- 
formation classes shown in Figure 9-33c. The final step 
in the process would be to recode the 10 information 
classes into just 6 classes to create a normal 6-class leg- 
end and to compute the hectares in each land-cover 
class. 
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Unsupervised Cluster Busting 


It is common when performing unsupervised classifica- 
tion using the chain algorithm or ISODATA to gener- 
ate n clusters (e.g., 100) and have no confidence in 
labeling g of them to an appropriate information class 
(let us say 30 in this example). This is because 1) the 
terrain within the IFOV of the sensor system contained 
at least two types of terrain, causing the pixel to exhibit 
spectral characteristics unlike either of the two terrain 
components, or 2) the distribution of the mean vectors 
generated during the unsupervised classification pro- 
cess was not good enough to partition certain impor- 
tant portions of feature space. When this occurs, it may 
be possible to perform cluster busting if in fact there is 
still some unextracted information of value in the re- 
mote sensing dataset. 


First, all the pixels associated with the g clusters (30 in 
a hypothetical example) that are difficult to label (e.g., 
mixed clusters 13, 22, 45, 92, etc.) are all recoded to a 
value of 1 and a binary mask file is created. A mask 
program is then run using 1) the binary mask file, and 
2) the original remote sensor data file. The output of 
the mask program is a new multiband image file con- 
sisting of only the pixels that could not be adequately 
labeled during the initial unsupervised classification. 
The analyst then performs a new unsupervised classifi- 
cation on this file, perhaps requesting an additional 25 
clusters. The analyst displays these clusters using stan- 
dard techniques and keeps as many of these new clus- 
ters as possible (e.g., 15). Usually, there are still some 
clusters that contain mixed pixels, but the proportion 
definitely goes down. The analyst may want to iterate 
the process one more time to see if an additional unsu- 
pervised classification breaks out additional clusters. 
Perhaps five good clusters are extracted during the final 
iteration. 


In this hypothetical example, the final cluster map 
would be composed of the 70 good clusters from the 
initial classification, 15 good clusters from the first 
cluster-busting pass (recoded as values 71 to 85), and 5 
from the second cluster-busting pass (recoded as values 
86 to 90). The final cluster map file may be put togeth- 
er using a simple GIS maximum dominate function. 
The final cluster map is then recoded to create the final 
classification map. 


er Fuzzy Classification 


Geographical information (including remotely sensed 
data) is imprecise, meaning that the boundaries be- 
tween different phenomena are fuzzy, or there is het- 


erogeneity within a class, perhaps due to differences in 
species, health, age, and so forth. 


For example, terrain in the southeastern United States 
often exhibits a gradual transition from water to for- 
ested wetland to deciduous upland forest as shown in 
Figure 9-34a. Normally, the greater the canopy clo- 
sure, the greater the amount of near-infrared energy re- 
flected from within the IFOV of a pixel along this 
continuum. Also, the greater the proportion of water in 
a pixel, the more near-infrared radiant flux absorbed. 
A hard classification algorithm applied to these re- 
motely sensed data collected along this continuum 
would be based on classical set theory, which requires 
precisely defined set boundaries for which an element 
(e.g., a pixel) is either a member (true = 1) or not a 
member (false = 0) of a given set. For example, if we 
made a classification map using just a single near-in- 
frared band (i.e., one-dimensional density slicing), the 
decision rules might be as shown in Figure 9-34a: 0 to 
30 = water, 31 to 60 = forested wetland, and 61 to 90 = 
upland forest. The classic approach creates three dis- 
crete classes with specific class ranges, and no interme- 
diate situations are allowed. Thus, using classical set 
theory, an unknown measurement vector may be as- 
signed to one and only one class (Figure 9-34a). But 
everyone knows that the phenomena grade into one 
another and that mixed pixels are present, especially 
around the values of 24 to 36 and 55 to 70, as shown in 
the figure. Clearly, there needs to be a way to make the 
classification algorithms more sensitive to the impre- 
cise (fuzzy) nature of the real world. 


Fuzzy set theory provides some useful tools for work- 
ing with imprecise data (Wang, 1990a,b; Lu and Weng, 
2007; Phillips et al., 2011). Fuzzy set theory is better 
suited for dealing with real-world problems than tradi- 
tional logic because most human reasoning is impre- 
cise and is based on the following logic. First, let XY be a 
universe whose elements are denoted x. That is, X¥ = 
{x}. As previously mentioned, membership in a classi- 
cal set A of X is often viewed as a binary characteristic 
function x 4 from X {0 or 1} such that x 4(x) = 1 if and 
only if x e A. Conversely, a fuzzy set Bin X is charac- 
terized by a membership function fp that associates with 
each x a real number from 0 to 1. The closer the value 
of f(x) is to 1, the more x belongs to B. Thus, a fuzzy 
set does not have sharply defined boundaries, and a set 
element (a pixel in our case) may have partial member- 
ship in several classes (Campbell and Wynne, 2011; 
Dronova et al., 2011). 


So how is fuzzy logic used to perform image classifica- 
tion? Figure 9-34b illustrates the use of fuzzy classifica- 
tion logic to discriminate among the three hypothetical 
land covers. The vertical boundary for water at bright- 
ness value 30 (Figure 9-34a) is replaced by a graded 
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FIGURE 9-34 a) Conventional hard classification rules applied to discriminate among three land-cover classes. The terrain 
icons suggest that there is a gradual transition in near-infrared brightness value as one progresses from water to forested wet- 
land to upland forest. A remote sensing system would be expected to record radiant flux from mixed pixels at the interface 
between the major land-cover types. Mixed pixels may also be encountered within a land-cover type as a result of differences 
in species, age, or functional health of vegetation. Despite these fuzzy conditions, a hard classification would simply assign a 
pixel to one and only one class. b) The logic of a fuzzy classification. In this hypothetical example, a pixel having a near-infra- 
red brightness value of <24 would have a membership grade value of 1.0 in water and 0 in both forested wetland and upland 
forest. Similarly, a brightness value of 60 would have a graded value of 0.70 for forested wetland, 0.30 for upland forest, and 
O for water. The membership grade values provide information on mixed pixels and may be used to classify the image using 


various types of logic. 


boundary that represents a gradual transition from wa- 
ter to forested wetland (Figure 9-34b). In the language 
of fuzzy set theory, BVs of less than 24 have a member- 
ship grade of 1 for water, and those greater than about 
70 have a membership grade of 1 for upland forest. At 
several other locations a BV may have a membership 
grade in two classes. For example, at BV 30 we have 
membership grades of 0.5 water and 0.5 of forested 
wetland. At BV 60 the membership grades are 0.7 for 
forested wetland and 0.3 for upland forest. This mem- 
bership grade information may be used by the analyst 
to create a variety of classification maps. 


Fuzzy set theory is not a panacea, but it does offer sig- 
nificant potential for extracting information on the 
makeup of the biophysical materials within a mixed 
pixel, a problem that will always be with us. Fortunate- 
ly, Spectral Mixture Analysis (SMA) can be used to ex- 
tract information from mixed pixels (e.g., Song, 2005; 
Lu and Weng, 2007). It evaluates each pixel spectrum 
as a linear combination of a set of endmember spectra 
(1.e., pure homogeneous materials such as water, as- 
phalt, closed vegetation canopy, bare soil) (Pu et al., 


2008; Colditz et al., 2011). The output of SMA is typi- 
cally presented in the form of fraction images, with one 
image for each endmember spectrum, representing the 
area proportions of the endmembers within the pixel. 
Thus, SMA provides fuzzy information. SMA is one of 
the most important hyperspectral image analysis func- 
tions and is described in detail in Chapter 11. 


)Object-Based Image Analysis 
\ (OBIA) Classification 


The twenty-first century has witnessed the develop- 
ment of remote sensing systems such as IKONOS, 
QuickBird, GeoEye-1, WorldView-2 and others that 
produce data with spatial resolutions of <1 X 1 m. Un- 
fortunately, classification algorithms based on single- 
pixel analysis often are not capable of extracting the in- 
formation desired from high spatial resolution remote 
sensor data (Pena-Barragan et al., 2011; Textron Sys- 
tems, 2014). For example, the spectral complexity of 
urban land-cover materials can result in limitations 
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when using per-pixel analysis for the classification of 
human-made materials such as roads and roofs and 
natural materials such as vegetation, soil, and water 
(Herold et al., 2002, 2003; Myint et al., 2011). Further- 
more, a significant but usually ignored problem with 
per-pixel characterization of land cover is that a sub- 
stantial proportion of the signal apparently coming 
from the land area represented by a pixel comes from 
the surrounding terrain (Townshend et al., 2000). Im- 
proved algorithms are needed that take into account 
not only the spectral characteristics of a single pixel 
but those of the surrounding (contextual) pixels. In ad- 
dition, we need information about the spatial charac- 
teristics of the surrounding pixels so that we can 
identify areas (or segments) of pixels that are homoge- 
neous (Frohn and Hao, 2006) and those that have 
changed (e.g., Im et al., 2008). 


Geographic Object-Based Image 
Analysis and Classification 


This need has given rise to the creation of geographic 
object-based image analysis (GEOBIA). The algo- 
rithms typically incorporate both spectral and spatial 
information in the image segmentation phase. The re- 
sult is the creation of image objects defined as individu- 
al areas with shape and spectral homogeneity (Benz, 
2001; Blaschke, 2010; Blaschke et al., 2014), which one 
may recognize as segments or patches in the landscape 
ecology literature (Frohn and Hao, 2006; Im et al., 
2008). In many instances, extracted image objects can 
provide a greater number of meaningful features for 
image classification (Wang et al., 2010; Liu and Xia, 
2010; Textron Systems, 2014). The objects can also be 
developed from any spatially distributed variable (e.g., 
elevation, slope, aspect, population density). Homoge- 
neous image objects are then analyzed using tradition- 
al classification algorithms (e.g., minimum distance, 
maximum likelihood), knowledge-based approaches 
and/or fuzzy classification logic. 


There are many algorithms that can be used to segment 
an image into relatively homogeneous image objects. 
Most can be grouped into two classes: edge-based al- 
gorithms and area-based algorithms. Unfortunately, 
the majority do not incorporate both spectral and spa- 
tial information, and very few have been used for re- 
mote sensing digital image classification. 


A very important approach to remote sensing image 
segmentation was developed by Baatz and Schape 
(2000). The image segmentation involves looking at in- 
dividual pixel values and their neighbors to compute a 
(Baatz et al., 2001): 


* color criterion (A,5/o,), and 
* a shape or spatial criterion (Asjap¢)- 


These two criteria are then used to create image objects 
(patches) of relatively homogeneous pixels in the re- 
mote sensing dataset using the general segmentation 
function (S,) (Baatz et al., 2001; Definiens, 2003, 2007): 


Sp ~ Woolor’ Beales =i We apices : Es jane 


(9.27) 
where the user-defined weight for spectral color versus 
shape is 0 < Weojor <1. If the user wants to place greater 
emphasis on the spectral (color) characteristics in the 
creation of homogeneous objects (patches) in the data- 
set, then W-.ojor is weighted more heavily (€.8., Weoior = 
0.8). Conversely, if the spatial characteristics of the da- 
taset are believed to be more important in the creation 
of the homogeneous patches, then shape should be 
weighted more heavily. 


Spectral (i.e., color) heterogeneity (A) of an image ob- 
ject is computed as the sum of the standard deviations 
of spectral values of each layer (o ;) (i.e., band) multi- 
plied by the weights for each layer (w;) (Kuehn et al., 
2002; Definiens, 2003): 


m 
h= y wp Of. 
k=1 


The col orcriterion is computed as the weighted mean 
of all changes in standard deviation for each channel k 
of the m band remote sensing dataset. The standard 
deviation o, are weighted by the object sizes n,, 
(Definiens, 2003): 


(9.28) 


m 


h= > wpltme 848 -(top1 Sp ton HDI (9.29) 
k=1 


where mg means merge. 


The shape criterion is computed using two landscape 
ecology metrics: compactness and smoothness. Hetero- 
geneity as deviation from a compact shape (cpt) is de- 
scribed by the ratio of the pixel perimeter length / and 
the square root of the number of pixels n forming an 
image object (i.e., a patch): 


SiS ce (9.30) 


ln 
Shape heterogeneity may also be described as smooth- 
ness, which is the ratio of the pixel perimeter length / 
and the shortest possible border length b of a box 
bounding the image object (i.e., a patch) parallel to the 
raster: 


(9.31) 


ols 


smooth = 
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The shape criterion incorporates these two measure- 
ments using the equation (Definiens, 2003, 2007): 


h =w,sh, +1 


bo Di (9.32) 


shape ~ Wis) Ph oneeih 
where 0 < wep; <1 is the user-defined weight for the 
compactness criterion. The change in shape heteroge- 
neity caused by each merge is evaluated by calculating 
the difference between the situation after and before 
image objects (ob) are merged. This results in the fol- 
lowing algorithms for computing roughness and 
smoothness (Definiens, 2003): 


Ing : loo1 
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"ng NI "obl AN "ob2 


(9.33) 
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mg 
where 7 is the object size in pixels. 


A pixel neighborhood function may be used to deter- 
mine whether an image object should be grown or 
whether a new image object should be created (Defini- 
ens, 2003). The logic associated with plane 4 and diag- 
onal 8 neighborhood functions is shown in Figure 9- 
35. In this example, the plane 4 neighborhood function 
results in the creation of two distinct image objects. 
The diagonal 8 neighborhood function results in a 
larger single-image object. 


The user specifies the spectral (color) and spatial shape 
parameters (compactness and smoothness) criteria and 
the neighborhood function logic (Definiens, 2007). A 
specially designed heuristic algorithm then applies 
these criteria to individual pixels in the scene and, in ef- 
fect, grows homogeneous regions (or, if you like, re- 
gions with specified amounts of heterogeneity). Once a 
segment patch exceeds the user-specified parameters, it 
stops growing. The final result is a new segmented im- 
age consisting of image objects (patches) that contain 
relatively homogeneous spectral and spatial character- 
istics (Jensen et al., 2006). 


Image Segmentation Classification Examples To 

appreciate the creation of such image objects (i.e., seg- 
ments or patches) and how they might be used to clas- 
sify a remotely sensed image, consider the high spatial 
resolution ADAR 5000 image of a yacht harbor on 
Pritchard’s Island, SC, in the Ace Basin (Figure 9-36). 
The data were obtained on September 23, 1999, at a 
spatial resolution of 0.7 % 0.7 m in four bands (blue, 
green, red, and near-infrared). Note the significant 
amount of spatial information in the dataset including 
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FIGURE 9-35 One of the criteria used to segment a re- 
motely sensed image into image objects is a pixel neigh- 
borhood function, which compares an image object being 
grown with adjacent pixels. The information is used to de- 
termine if the adjacent pixel should be merged with the ex- 
isting image object or be part of a new image object. a) In 
this example, if a plane 4 neighborhood function is select- 
ed, then two image objects would be created because the 
pixels under investigation are not connected at their plane 
borders. b) Pixels and objects are defined as neighbors in a 
diagonal 8 neighborhood if they are connected at a plane 
border or a corner point (Definiens, 2003). In this example, 
image object 1 can be expanded because it connects at a 
diagonal corner point. This resulted is a larger image object 
1. Other types of neighborhood functions could be used. 


building rooftops oriented toward and away from the 
Sun, automobiles on the concrete or asphalt pavement, 
upland forest texture, shadows, and Smooth Cordgrass 
(Spartina alterniflora) wetland along the major tidal 
tributaries. 


The object-based image segmentation process was 
weighted so that spectral information (color) was more 
important than spatial (shape) information (Le., Weojop 
= 0.8). Three spectral bands of ADAR 5000 data 
(green, red, and near-infrared) were used in the seg- 
mentation process. The spatial (shape) parameter was 
more heavily weighted to smoothness (0.9) than com- 
pactness (0.1). The user-specified inputs were then 
used to identify various “scales” of image segmenta- 
tion wherein each successive scale file contained larger 
segmented image objects. This process is commonly re- 
ferred to as multiresolution image segmentation (Baatz 
and Schape, 2000; Definiens, 2007). Figure 9-36 shows 
multiresolution image objects associated with four lev- 
els of aggregation or scale: 10, 20, 30, and 40. Each im- 
age object (i.e, patch, segment, or polygon, if you 
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Image Segmentation Based on Spectral (Green, Red, Near-infrared) 
and Spatial (Smoothness and © mpactness) G iteria 


c. Segmentation scale 30. 


d. Segmentation scale 40. 


Color composite of ADAR 5000 imagery of a yacht harbor on Pritchard's Island, SC, in the Ace Basin obtained 


on n September 23, 1999, at a spatial resolution of 0.7 H 0.7 m. Multiresolution image segmentation was performed using 
three bands (green, red, and near-infrared) at four segmentation scales: 10, 20, 30, and 40. The segmentation process was 
weighted so that spectral information (color) was more important than spatial (shape) information (weighted 0.8 to 0.2, re- 
spectively). The spatial parameter was more heavily weighted to smoothness (0.9) than compactness (0.1). 


prefer) in each segmentation scale is hierarchically re- 
lated to all other files. In other words, the boundaries 
of one large polygon at segmentation scale 40 may be 
topologically linked to perhaps four smaller polygons 
at segmentation scale 10. The four smaller polygons at 
scale 10 will have exactly the same outer-perimeter co- 
ordinates as the larger polygon at scale 40. 


One of the most important aspects of performing this 
type of image segmentation using remote sensor data is 
that each image object in the dataset (i.e., each poly- 
gon) contains not only the mean spectral values of all 
pixels found within each band but also various spatial 
measurements that characterize the shape of the poly- 
gon. Table 9-12 summarizes several image object met- 
rics that may be computed for each polygon. 
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OBIA ClassiXcation of a Yacht Harbor on Pritchard’s Island, SC, at Four Segne ntation Scales 


a 


c. Segmentation scale 30. 


Water 
Bare Soil 
Shadow 


impervious Surfaces 
Concrete/asphalt 
Roof in direct sunlight 


d. Segmentation scale 40. 


Upland Vegetation 
Deciduous forest (Quercus spp.) 
Grass 


Wetland 
Smooth cordgrass (Spartina alterni&ora) 
Other wetland 


Roof oriented away from direct sunlight Exposed mud Kat 


FIGURE 9-37 Four OBIA classification maps derived from four different sets of image segmentation scale data of a yacht 


harbor on Pritchard's Island, SC, in the Ace Basin. 


The OBIA classification of a segmented image is sub- 
stantially different from performing a per-pixel classifi- 
cation (Lu and Weng, 2007; Duro et al., 2012). First, 
the analyst is not constrained to using just spectral in- 
formation. He or she may choose to use a) the mean 


spectral information in conjunction with b) various 
shape measures associated with each image object 
(polygon) in the dataset (Definiens, 2007). This intro- 
duces flexibility and robustness. Once selected, the 
spectral and spatial attributes of each polygon may be 
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TABLE 9-12 A selection of various landscape-ecology metrics that can be computed from individual image objects (patches) 
that have been segmented within an image (based on O'Neill et al., 1997; Frohn and Hao, 2006; Jensen, 2007). 


of an Image 


Image Object Algorithm Description 
Metrics 
Z Image object (i.e., polygon, patch) mean value (M,) in a single layer is 
» Mi calculated by summing all the pixel values within the image object (v,) 
Mean = and dividing by the total number of pixels in the object, n. If the image 
Up = — object is composed of spectral brightness values, then v; = BV;. Other- 
it wise, v; may be any type of data (e.g., radar backscatter, LIDAR eleva- 
tion value, DEM elevation value). 
n 
For a single image object (i.e., polygon, patch), the sum of all mean lay- 
Spectral » H; er values (X) divided by the total number of layers (nj) (i-e., bands). 
Mean p= i=l 
Ny, 
Ly The mean of a single image object (4) divided by the sum of all spec- 
Spectral Ratio ratio; = tral layers (bands) associated w th this image obj ect (X)). 


Object y Hj 
,= 
Standard ny, For a single image object (i.e., polygon, patch) in a single layer, the sum 
Deviation 2 of all pixel layer values (v;) subtracted from the mean of the image ob- 
of an Image » 0; im Hy) ject (,) squared, divided by the total number of pixels in the polygon 
Object Ne (n) minus one. 
(oy —=} 
L n—1 
Mean For a single image object (i.e., polygon, patch), the mean difference to 
Difference 1 ae a direct neighbor is calculated using | = border length in pixels of the 
to Neighbor- DN pe yy 1 (= Uy) image object, |,; = border length shared with a direct (attached neigh- 
ing Image i bor), (;) = image object mean value of layer i, (X; )) = neighboring im- 
Object G= Il age obj ect mean value of layer i, and nn = number of neighbors. 
Length-to- ] The length-to-width ratio (y ) is computed by dividing the length (/) by 
Width Ratio VS = the width (w) of an image object. This is approximated by determining 
Ve the dimensions of the smallest box that bounds the polygon. 
n 
For georeferenced data, the area of an image object (A) equals the 
Area A= y a; summation of the true area (a;) of each of the rn pixels in the image ob- 
ject. An image obj ect consisting of six Landsat Thematic Mapper 30 & 
a= Il 30 m pixels would have an area of 6 pixels H 90m? = 540 m?. 
Length of an image object is approximated using the length-to-width 
Length 1 — VA-y ratio. If the image object has a curved shape, it is better to break it into 
sub-subjects. 
A Width of an image object is approximated using the length-to-width ra- 
Width w= ic tio. If the image object has a curved shape, it is better to break it into 
¥ sub-objects. 
Border L The border length of an image object is the sum of the number of edg- 
Length bf = S e; es (e;) that touch all neighboring image objects (or that touch the out- 


side edge of a scene). 
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TABLE 9-12 A selection of various landscape-ecology metrics that can be computed from individual image objects (patches) 
that have been segmented within an image (O'Neill et al., 1997; Frohn and Hao, 2006; Jensen, 2007). 


Image Object Algorithm Description 
Metrics 
Shape Index A The shape index of an image object is the border length (bl) divided by 
ST = four times the square root of the area (A). The smoother the shape, the 
4x na lower the value. The greater the value, the more fractal the shape. 
Density s/n The density of an image object is the area of the object divided by its 
= _ radius. It is approximated by computing the variance of all x- and y- 
1+ /Var(X)+ Var(Y) | coordinates of all n pixels forming the image object. Density is a surro- 
gate for compactness. The more compact the object, the higher its 
density and the more the shape is like a square. 
Asymmetry i Image object asymmetry is one minus the ratio of the length of the mi- 
k= 1-— nor (n) and major (m) axes of an ellipse enclosing the image object. The 
ue greater the asymmetry, the greater the value. 
Classified 
Map Summary 
Metrics 


Dominance 


D 


n 
=a y 


(-P,- InP) 
In(n) 


Dominance (0 < D < 1) identifies the extent to which the landscape is 

dominated by a single land-cover type where 0 < P; < 1 is the propor- 
tion of land-cover type k and n is the total number of land-cover types 
in the land-cover map. 


k=1 
alee (Pj nP;,) Contagion (0 < C < 1) expresses the probability that land cover is more 
Contagion =1- Detar 5 FIGS clumped than the random expectation where P;;is the probability that a 
pixel of cover type jis adjacent to type j. 
The fractal dimension, F, of patches (image objects) indicates the extent 
Fractal Ter of human reshaping of the landscape. It is calculated by regressing the 
Dimension log of the patch perimeter against the log of the patch area for each 


patch on the landscape. The index equals twice the slope of the regres- 


sion line. Patches < 4 pixels are excluded. 


input to a variety of classification algorithms for analy- 
sis (e.g., parallelepiped, nearest-neighbor, minimum 
distance, maximum likelihood). 


The classification process is usually very fast because 
individual image objects (i.e., polygons) rather than in- 
dividual pixels are assigned to specific classes. Consider 
the Pritchard’s Island example where the dataset con- 
sists of 543 rows and 460 columns. A per-pixel classifi- 
cation would process 249,780 pixels. Figure 9-37a 
depicts the object-oriented classification of just 2,391 
polygons into nine classes using segmentation scale 10 
data. Only 753 image objects (polygons) were classified 
in Figure 9-37b (segmentation scale 20 data); 414 in 
Figure 9-37c (segmentation scale 30 data), and 275 in 
Figure 9-37d (segmentation scale 40 data). 


Examination of Figure 9-37 reveals some interesting 
aspects of OBIA classification. First, the user must de- 
cide what level of polygon aggregation is acceptable for 
the task at hand. In this example, a classification based 
on the use of segmentation scale 20 data (Figure 9-37b) 
is probably the best representation. It provides the 
greatest amount of accurate information for both very 
small objects and larger, geographically-extensive sur- 
face materials such as forest and wetland. Conversely, 
if only segmentation scale 40 data (Figure 9-37d) were 
used, then the classification of the wetland would con- 
tain serious error. Perhaps the information that the 
user wants is best extracted using a combination of in- 
formation present in image segmentation scale 10 for 
relatively small rooftop materials and segmentation 
scale 40 for identifying homogeneous forest areas. The 
user must decide which of the various segmented files 
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a. Color-infrared aerial photography (1 x | ft. resolution). 


c. Classiication of individual objects. 


Object-based Image Analysis (OBIA) of Estuarine Intertidal Habitat near Bluffton, SC 


b. Segmentation scale 20. 


Legend Class 

Buildings 

Dock 

Exposed MudXat 

Grass 

Juncus 

Salicornia/Harbottom 

Shadow 

Shrub-Scrub/Forest 

Tall Creekside Spartina alterni€ora 


Intermediate Spartina alternikora 
| Water 


FIGURE 9-38 a) Digital frame camera color-infrared aerial photography obtained in 2006 at 1X 1 ft. spatial resolution. b) 
OBIA image segmentation at a scale of 20. c) Classification of the individual objects according to the U.S. Fish & Wildlife Ser- 
vice (USFS) Cowardin et al. (1979) estuarine intertidal habitat classification scheme summarized in Figures 9-5 and 9-6. 


are of most value. He or she can then enter very specif- 
ic rules into the classification process if desired 


A second example provides a more detailed examina- 
tion of estuarine marine habitat using 2006 high spatial 
resolution digital camera aerial photography (1 X 1 ft.) 
for an area near Bluffton, SC (Figure 9-38). After ex- 
perimentation, a segmentation scale of 20 was judged 
to be the best level to create useful polygons associated 
with the various wetland land-cover types. Color was 
once again more important than spatial (shape) infor- 
mation (1.€., Weojor = 9.8) and the spatial (shape) pa- 
rameter was more heavily weighted to smoothness (0.9) 
than compactness (0.1). The supervised OBIA classifi- 
cation provides detailed wetland information with 
classes derived from the U.S. Fish & Wildlife Service 
Cowardin et al. (1979) wetland classification scheme 
summarized in Figures 9-5 and 9-6. 


OBIA Classification Considerations 


Once the object-based image segmentation and classifi- 
cation is complete, it may be desirable to evaluate the 
characteristics of all the patches (image objects) in the 
map to address important applications. O’ Neill (1997) 
suggested that the functional health of an ecosystem 
can be monitored if the following three landscape pat- 
tern and structure metrics (indices) are monitored 
through time: dominance, contagion, and fractal di- 
mension (Table 9-12). Other studies use different land- 
scape pattern and structure metrics (Batistella et al., 
2003; Frohn and Hao, 2006; Jensen et al., 2006). 


Per-pixel classifications often appear pixelated. Con- 
versely, object-oriented classification can appear frac- 
tal. The analyst conducting an _ object-oriented 
classification should decide what level (scale) of seg- 
mentation is most acceptable for the task at hand and 
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TABLE 9-13 Selected image segmentation freeware (accessible in 2013). 


Segmentation Program Developer 


Location 


Efficient Graph-Based 
Image Segmentation 


Felzenszwalb, P.F. and D. P. Huttenlo- 
cher, 2004; (University of Chicago) 


http://www.cs.brown.edu/~pff/segment/. 


JSEG - Segmentation of 
Color-Texture Regions 


Deng, Y. and Manjunath, 2001; (Univer- 
sity of California at Santa Barbara) 


http://vision.ece.ucsb.edu/segmentation/ 
jseg/. 


Multiscale Normalized Cuts 
Segmentation Toolbox 
for MATLAB 


Cour, T., Benezit, F. and J. Shi, 2005; 
(University of Pennsylvania) 


http://www.timotheecour.com/software/ 
ncut_multiscale/ncut_multiscale.html. 


SPRING - Georeferenced 
Information Processing 
System 


Research, 2014 


Brazilian National Institute for Space 


http://www.dpi.inpe.br/spring/english/in- 
dex. html. 


most easily understood by lay viewers when the final 
thematic map(s) are produced. 


Segmentation into image objects allows the use of geo- 
graphical and landscape ecology concepts involving 
neighborhood, distance, and location for analyzing re- 
motely sensed data. It also facilitates the merging or 
fusion of multiple types of remote sensor data for a 
particular application (Kuehn et al., 2002). Object- 
based image segmentation and classification is a major 
paradigm shift when compared with per-pixel classifi- 
cation (Lu and Weng, 2007; Blaschke et al., 2014). It 
will become increasingly important not only for single- 
date image classification but also for change detection 
(e.g., Im, 2006; Im et al., 2008; Campbell, 2010). 


For example, Jensen et al. (2006) extracted very accu- 
rate agricultural land-cover information using OBIA 
applied to SPOT multispectral data. The land-cover in- 
formation was used to model agricultural water de- 
mand in South Africa. Hamilton et al. (2007) 
developed an OBIA-based forest stand delineation sys- 
tem for the U.S. Dept. of Agriculture Forest Service. 
Hofmann et al. (2008) used OBIA applied to Quick- 
bird data to inventory informal settlements in Rio de 
Janeiro. Tsai et al. (2011) used OBIA to extract build- 
ing footprints from high spatial resolution imagery of 
Accra, Ghana. Myint et al. (2011) found OBIA yielded 
more accurate results for classifying urban land cover 
from high spatial resolution imagery than traditional 
per-pixel classifiers such as maximum likelihood. Duro 
et al. (2012) compared pixel-based and object-based 
image analysis approaches for classifying broad land- 
cover classes over agricultural landscapes. Interesting- 
ly, they found no significant difference in the results 
obtained using OBIA when compared with three pixel- 
based supervised machine learning algorithms dis- 


cussed in the next chapter (decision tree, random for- 
est, and the support vector machine). 


The following selected commercial digital image pro- 
cessing software has OBIA capability: Definiens 
eCognition; Intergraph ERDAS Imagine; Trimble EN- 
VI; and Clark Lab IDRISI. Selected sources of image 
segmentation freeware are summarized in Table 9-13 
(updated from Hamilton et al., 2007). 


Incorporating Ancillary Data 
*. in the Classification Process 


An analyst photo-interpreting a color aerial photo- 
graph of the terrain often has at his or her disposal 1) 
systematic knowledge about the soils, geology, vegeta- 
tion, hydrology, and geography of the area, 2) the abili- 
ty to visualize and comprehend the landscape’s color, 
texture, height, and shadows, 3) the ability to place 
much of this diverse information in context to under- 
stand site conditions and associations among phenom- 
ena, and 4) historical knowledge about the area. 
Conversely, 95% of all remote sensing digital image 
classifications attempt to accomplish the same task us- 
ing a single variable, i.e., an object’s spectral reflectance 
characteristics (color) or black-and-white tone. There- 
fore, it is not surprising that there is error in remote 
sensing—derived classification maps. Why should we ex- 
pect the maps to be extremely accurate when the infor- 
mation provided to the classification algorithm is so 
rudimentary? 


Numerous scientists recognize this condition and have 
attempted to improve the accuracy and quality of re- 
mote sensing—derived land-cover classification by in- 
corporating ancillary data in the classification process 
(e.g., Hutchinson, 1982; McIver and Friedl, 2002; Lu 
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and Weng, 2007). Ancillary data are any type of spatial 
or nonspatial information that may be of value in the 
image classification process, including elevation, slope, 
aspect, geology, soils, hydrology, transportation net- 
works, political boundaries, and vegetation maps. An- 
cillary data are not without error. Analysts who want 
to incorporate ancillary data into the classification 
process should be aware of several considerations. 


Problems Associated with Ancillary 
Data 


First, ancillary data were usually produced for a specif- 
ic purpose and it was not to improve remote sensing 
classification accuracy. Second, the nominal, ordinal, 
or interval thematic attributes on the collateral maps 
may be inaccurate or incomplete. Great care must be 
exercised when generalizing the classes found on the 
ancillary map source materials as we try to make them 
compatible with the remote sensing investigation class- 
es of interest. 


Third, considerable ancillary information is stored in 
analog map format. The maps must be digitized, trans- 
lated, rotated, rescaled, and often resampled to bring 
the dataset into congruence with the remote sensing 
map projection. During this process the locational at- 
tributes of the phenomena may be moved from their 
true planimetric positions. This assumes that the ancil- 
lary data were planimetrically accurate to begin with. 
Unfortunately, considerable ancillary data were never 
recorded in their proper planimetric positions. For ex- 
ample, old soil surveys published by the U.S. Soil Con- 
servation Service were compiled onto uncontrolled 
photomosaics. Analysts trying to use such data must 
be careful that they do not introduce more error into 
the classification process than they are attempting to 
remove. 


Approaches to Incorporating 
Ancillary Data to Improve Remote 
Sensing Classification Maps 


Several approaches may be used to incorporate ancil- 
lary data in the image classification process that should 
improve results. These include incorporating the data 
before, during, or after classification through geo- 
graphical stratification, classifier operations, and/or 
post-classification sorting. Ancillary data may also be 
incorporated using object-oriented image segmenta- 
tion, neural networks, expert systems, and decision- 
tree classifiers. 


Geographical Stratification 
Ancillary data may be used prior to classification to 
subdivide the regional image into strata, which may 


then be processed independently. The goal is to in- 
crease the homogeneity of the individual stratified im- 
age datasets to be classified. For example, what if we 
wanted to locate spruce fir in the Colorado Rockies 
but often encountered misclassification up and down 
the mountainside? One approach would be to stratify 
the scene into just two files: one with elevations from 0 
to 2,600 ft above sea level (dataset 1) and another with 
elevation > 2,600 ft ASL (dataset 2). We would then 
classify the two datasets independently. Spruce fir do 
not grow below 2,600 ft ASL; therefore, during the 
classification process we would not label any of the pix- 
els in dataset | spruce fir. This would keep spruce fir 
pixels from being assigned to forested areas that can- 
not ecologically support them. Errors of commission 
for spruce fir should be reduced when datasets 1 and 2 
are put back together to compile the final map and are 
compared to a traditional classification. If specific eco- 
logical principles are known, the analyst could stratify 
the area further using slope and aspect criteria to refine 
the classification. 


Stratification is a conceptually simple tool and, care- 
fully used, can be effective in improving classification 
accuracy. logical stratification can have severe impli- 
cations. For example, differences in training set selec- 
tion for individual strata and/or the vagaries of 
clustering algorithms, if used, may produce different 
spectral classes on either side of strata boundaries. 
Edge-matching problems become apparent when the 
final classification map is put together from the maps 
derived from the individual strata. 


Classifier Operations 

Several methods may be used to incorporate ancillary 
data during the image classification process. One of the 
most useful is the logical channel method. A per-pixel 
logical channel classification includes ancillary data as 
one of the channels (features) used by the classification 
algorithm. For example, a dataset might consist of 
three IKONOS bands of spectral data plus two addi- 
tional bands (percent slope and aspect) derived from a 
digital elevation model. The entire five-band dataset is 
acted on by the classification algorithm. When ancil- 
lary data are incorporated into traditional classifica- 
tion algorithms as logical channels, the full range of 
information available in the ancillary data is used (e.g., 
Ricchetti, 2000). With logical channel addition, ancil- 
lary data are given equal weight to single spectral 
bands unless weights are assigned in a maximum likeli- 
hood classifier (Lawrence and Wright, 2001). Chen and 
Stow (2003) used the logical channel approach to clas- 
sify land cover using multiple types of imagery ob- 
tained at different spatial resolutions. 


The context of a pixel refers to its spatial relationship 
with any other pixel or group of pixels throughout the 


CHAPTER 9 THEMATIC INFORMATION EXTRACTION: PATTERN RECOGNITION 423 


scene. Contextual logical channel classification occurs 
when information about the neighboring (surround- 
ing) pixels is used as one of the features in the classifi- 
cation. Texture is one simple contextual measure that 
may be extracted from an n window (see Chapter 8) 
and then added to the original image dataset prior to 
classification (Stow et al., 2003). It is important to re- 
member that contextual information may also be de- 
rived from nonimage ancillary sources, such as maps 
showing proximity to roads, streams, and so on. 


A second approach involves the use of a priori proba- 
bilities in the classification algorithm. The analyst gets 
the a priori probabilities by evaluating historical sum- 
maries of the region (e.g., last year cotton accounted 
for 80% of the acreage, hay 15%, and barley 5%). 
These statistics can be incorporated directly into a 
maximum likelihood classification algorithm as 
weights p(w;) to the classes, as previously discussed. 
Prior probabilities can improve classification results by 
helping to resolve confusion among classes that are 
difficult to separate and by reducing bias when the 
training sample is not representative of the population 
being classified. McIver and Friedl (2002) point out 
that the use of a priori probabilities in maximum likeli- 
hood classification is often problematic in practice. 
They developed a useful method for incorporating pri- 
or probabilities into a nonparametric decision-tree 
classifier. 


Another approach involves image segmentation, as dis- 
cussed in the previous section. This method can incor- 
porate both spectral and nonspectral ancillary data, 
which are subjected to multiresolution segmentation to 
produce polygons (patches) that contain relatively uni- 
form spectral and spatial characteristics. This is a 
straightforward way of merging both spectral and non- 
spectral information. 


Ancillary data have been incorporated into modern 
classification methods such as expert systems and neu- 
ral networks (Stow et al., 2003; Qiu and Jensen, 2004). 
These approaches incorporate the ancillary data direct- 
ly into the classification algorithms and are usually not 
dependent on a priori weights. Chapter 10 describes 
how such systems work and the ease with which ancil- 
lary data may be introduced, including the fact that 
they are not confined to normally distributed data. 


Machine-learning approaches have been used to estab- 
lish rule-based classification systems where expert 
knowledge was inadequate (Huang and Jensen, 1997; 
Myint et al., 2011). Lawrence and Wright (2001) used 
rule-based classification systems based on classifica- 
tion and regression tree (CART) analysis to incorpo- 
rate ancillary data into the classification process. 


Post-Classification Sorting 

This method involves the application of very specific 
rules to (1) initial remote sensing classification results 
and (2) spatially distributed ancillary information. For 
example, Hutchinson (1982) classified Landsat MSS 
data of a desert area in California into nine initial 
classes. He then registered slope and aspect maps de- 
rived from a digital elevation model with the classifica- 
tion map and applied 20 if—then rules to the datasets 
(e.g., if the pixel was initially classified as an active 
sand dune and if the slope <1%, then the pixel is a dry 
lake bed). This eliminated confusion between several 
of the more prominent classes in this region (e.g., be- 
tween the bright surfaces of a dry lake bed [playa] and 
the steep sunny slopes of large sand dunes). Similarly, 
Cibula and Nyquist (1987) used post-classification 
sorting to improve the classification of Landsat MSS 
data for Olympic National Park. Topographic (eleva- 
tion, slope, and aspect) and watershed boundary data 
(precipitation and temperature) were analyzed in con- 
junction with the initial land-cover classification using 
Boolean logic. The result was a 21-class forest map 
that was just as accurate as the initial map but con- 
tained much more information. 


The incorporation of ancillary data in the remote sens- 
ing classification process is important. However, the 
choice of variables to include is critical. Common sense 
suggests that the analyst thoughtfully select only vari- 
ables with conceptual and practical significance to the 
classification problem at hand. Incorporating illogical 
or suspect ancillary information can rapidly consume 
limited data analysis resources and lead to inaccurate 
results. 
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INFORMATION EXTRACTION USING 
ARTIFICIAL INTELLIGENCE 


One definition of artificial intelligence (AI) is: 


the theory and development of computer 
systems able to perform tasks normally re- 
quiring human intelligence (Oxford, 2014). 


But how do we know when an artificially intelligent 
system has been created? Ideally we could use the Tur- 
ing test, which suggests that if we are unable to distin- 
guish a computer’s response to a problem of interest 
from a human’s response to the same problem, then the 
computer system is said to have intelligence (Turing, 
1950). The test or task is for an artificial intelligence 
program to have a blind conversation with an interro- 
gator for 5 minutes. The interrogator has to guess if the 
conversation is with an artificial intelligence program 
or with a real person. The AI program passes the test if 
it fools the interrogator 30% of the time. Unfortunate- 
ly, it is very difficult for most artificial intelligence sys- 
tems to pass the Turing test. For this reason, “the field 
of AI as a whole has paid little attention to Turing 
tests,” preferring instead to forge ahead developing ar- 
tificial intelligence applications that simply work (Rus- 
sell and Norvig, 2010). 


Artificial intelligence research was initiated in 1955 
when Allen Newell and Herbert Simon at the RAND 
Corporation proved that computers could do more 
than calculate. 


They demonstrated that computers were 
physical symbol systems whose symbols 
could be made to stand for anything, includ- 
ing features of the real world, and whose 
programs could be used as rules for relating 
these features. In this way computers could 


be used to simulate certain important as- 
pects of intelligence. Thus, the information- 
processing model of the mind was born 
(Dreyfus and Dreyfus, 2001). 


Unfortunately, artificial intelligence was oversold in 
the 1960s much like remote sensing was oversold in the 
1970s. General artificial intelligence problem solving 
was found to be much more difficult than originally 
suggested. Scientists had difficulty getting computers 
to solve problems that were routinely solved by human 
experts. Therefore, scientists instead started to investi- 
gate the development of artificial intelligence applica- 
tions in “micro-worlds,” or very narrow topical areas. 
This led to the creation of the first useful artificial in- 
telligence systems for select applications, e.g., games, 
disease diagnosis (MYCIN), spectrograph analysis 
(DENDRAL). MYCIN was developed at Stanford 
University in 1976 to aid physicians in diagnosing and 
treating patients with infectious blood diseases caused 
by bacteria in the blood and meningitis. These diseases 
can be fatal if not recognized and treated quickly. The 
DENDRAL program solved the problem of inferring 
molecular structure from the information provided by 
a mass spectrometer (Buchanan and Lederberg, 1971). 
DENDRAL was the first truly successful knowledge- 
intensive system deriving its expertise from large num- 
bers of special-purpose rules. The field advanced at a 
rapid rate. For example, NASA’s REMOTE AGENT 
program became the first on-board autonomous plan- 
ning program to control the scheduling of operations 
for a spacecraft travelling a hundred million miles from 
Earth (Jonsson et al., 2000). Such expert systems are 
based on the use of knowledge or rules derived from 
human experts. The knowledge is extracted from the 
expert by a knowledge engineer and turned into useful 
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rule-based reasoning that can, hopefully, be performed 
by a computer. 


cia Overview 


This chapter introduces the fundamental characteris- 
tics of several knowledge-based expert systems includ- 
ing machine-learning derived decision trees and 
regression trees, random forest decision-tree analysis 
and support vector machines. It then describes the 
characteristics of neural networks and their advantages 
and limitations when used to extract information from 
remotely sense images. 


a Expert Systems 


A knowledge-based expert system may be defined as “a 
system that uses human knowledge to solve problems 
that normally would require human intelligence” (PC 
AI, 2002). It uses artificial intelligence techniques in 
problem-solving processes to support human decision- 
making, learning, and action (Akerkar and Sajja, 
2009). It is the ability to solve problems efficiently and 
effectively in a relatively narrow problem area and to 
perform at the level of an expert. Expert systems repre- 
sent the expert’s domain (i.e., subject matter) knowl- 
edge base as data and rules within the computer. The 
rules and data can be called upon when needed to solve 
problems. A completely different problem within the 
domain of the knowledge base can be solved without 
having to reprogram. 


Knowledge-based expert systems are used extensively 
in remote sensing research and production. Machine- 
learning decision-tree classifiers may be used to extract 
categorical information such as land use/land cover. 
For example, decision-tree classification is used to cre- 
ate the NASA global MODIS Collection 5 Land Cover 
type product (Fried et al., 2010). Both Pena-Barragan 
et al. (2011) and Duro et al. (2012) used various deci- 
sion trees to classify agriculture land cover. Machine- 
learning regression trees can also be used to predict 
continuous quantitative information. For example, 
Coops et al. (2006) used classification and regression 
tree (CART) analysis to map the probability of Moun- 
tain Pine Beetle infestation in Canada. Im et al. 
(2012ab) used regression trees to predict LAI and per- 
cent impervious surface. 


Expert systems may also be used to detect change in 
complex heterogeneous environments (Yang and 
Chung, 2002). For example, Im and Jensen (2005) in- 
troduced local correlation image analysis for change 


detection and created neighborhood correlation imag- 
es (NCIs) associated with bi-temporal high spatial res- 
olution datasets. They used a machine learning 
decision-tree classifier (C5.0) to extract change infor- 
mation from the bi-temporal data. The USGS uses de- 
cision-tree and regression-tree classifiers to detect 
change associated with its National Land Cover Data- 
base (NLCD) products (Xian et al., 2009, 2011). 


A knowledge-based expert system consists of the com- 
ponents shown in Figure 10-1, including: 


¢ human expert, 

* user interface, 

¢ knowledge base (rule-based domain), 
¢ inference engine, 

¢ on-line databases, and 

* user. 


It is useful to review the characteristics of each of these 
components. 


Expert System User Interface 


The expert system user interface should be easy to use, 
interactive, and interesting. It should also be intelligent 
and accumulate user preferences in an attempt to pro- 
vide the most pleasing communication environment 
possible. Figure 10-2 depicts a commercially available 
Knowledge Engineer interface that can be used to de- 
velop remote sensing—assisted expert systems. This ex- 
pert system she/l was built using object-oriented 
programming and is easy to use. All of the hypotheses, 
rules, and conditions for an entire expert system may 
be viewed and queried from the single user interface. 


Creating the Knowledge Base 


Images, books, articles, manuals, and periodicals have 
a tremendous amount of information in them. Practi- 
cal experience in the field with vegetation, soils, rocks, 
water, atmosphere, and urban infrastructure is also tre- 
mendously important. However, a human must com- 
prehend the information and experiences and turn it 
into knowledge for it to be useful. Many human beings 
have trouble interpreting and understanding the infor- 
mation in images, books, articles, manuals, and period- 
icals. Similarly, some do not obtain much knowledge 
from field work. Fortunately, some laypersons and sci- 
entists are particularly adept at processing their knowl- 
edge using three different problem-solving approaches: 


¢ algorithms using conventional computer programs, 
¢ heuristic knowledge-based expert systems, and 
¢ artificial neural networks. 
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Interface 


Expert is the 
Knowledge Base 
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Domain Knowledge 
- Hypotheses 

- Rules 

- Conditions 
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Inference Engine 
- Solves problems 
- Draws conclusions 


User 


Online Databases 


FIGURE 10-1 The components of a typical rule-based expert system. The domain (thematic) knowledge contained in an ex- 
pert’s mind is extracted in the form of a knowledge base that consists of hypotheses (problems), rules, and conditions that 

satisfy the rules. A user interface and an inference engine are used to encode the knowledge base rules, extract the required 
information from online databases, and solve problems. Hopefully, the information is of value to the user who queries the ex- 


pert system. 


Algorithmic Approaches to Problem Solving 
Conventional algorithmic computer programs contain 
little knowledge other than the basic algorithm for 
solving a specific problem, the necessary boundary 
conditions, and data. The knowledge is usually embed- 
ded in the programming code. As new knowledge be- 
comes available, the program has to be changed and 
recompiled (Table 10-1). 


Heuristic Knowledge-Based Expert System 
Approaches to Problem Solving 
Knowledge-based expert systems, on the other hand, 
collect many small fragments of human know-how for 
a specific application area (domain) and place them in 


a knowledge base that is used to reason through a prob- 
lem, using the knowledge that is most appropriate. Im- 
portant characteristics that distinguish knowledge- 
based expert systems from conventional algorithmic 
systems are summarized in Table 10-1. Heuristic knowl- 
edge may be defined as “involving or serving as an aid 
to learning, discovery, or problem solving by experi- 
mental and especially by trial-and-error methods. Heu- 
ristic computer programs often utilize exploratory 
problem-solving and self-educating techniques (as the 
evaluation of feedback) to improve performance” 
(Merriam-Webster, 2003). 


The Problem with Experts Unfortunately, most ex- 
perts really do not know exactly how they perform 
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FIGURE 10-2 The Knowledge Engineer interface used in ERDAS Imagine’s Expert Classifier (interface courtesy of Hexagon 


Geospatial). 


their expert work (Dreyfus and Dreyfus, 2001). Much 
of their expertise is derived from experiencing life and 
observing hundreds or even thousands of case studies. 
It is often difficult for experts to understand the intri- 
cate workings of complex systems much less be able to 
break them down into their constituent parts and then 
mimic the decision-making process of the human 
mind. Therefore, how is it possible to get the knowIl- 
edge embedded in the mind of an expert into the for- 
mal rules and conditions necessary to create an expert 
system to solve relatively narrowly defined hypotheses 
(i.e., problems)? This is the primary responsibility of 
the knowledge engineer. 


The knowledge engineer interrogates the domain ex- 
pert and extracts as many rules and conditions as pos- 
sible that are relevant to the hypotheses (problems) 
being examined. Ideally, the knowledge engineer has 
unique capabilities that allow him or her to help build 
the most appropriate rules. This is not easy. The 
knowledge engineering process can be costly and time- 
consuming. 


Recently, it has become acceptable for a domain expert 
(e.g., biologist, geographer, agronomist, forester) to 
create his or her own knowledge-based expert system 
by querying oneself and hopefully accurately specify- 
ing the rules associated with the problem at hand, for 
example, using Hexagon’s ERDAS Imagine expert sys- 
tem Knowledge Engineer (Hexagon ERDAS Imagine, 
2014). When this activity takes place, the expert must 
have a wealth of knowledge in a certain domain and 
the ability to formulate a hypothesis and parse the 


rules and conditions into understandable elements that 
are very amenable to the “knowledge representation 
process.” 


The Knowledge Representation Process 

The knowledge representation process normally in- 
volves encoding information from verbal descriptions, 
rules of thumb, images, books, maps, charts, tables, 
graphs, equations, etc. Hopefully, the knowledge base 
contains sufficient high-quality rules to solve the prob- 
lem under investigation. Rules are normally expressed 
in the form of one or more “IF condition THEN ac- 
tion” statements. The condition portion of a rule state- 
ment is usually a fact, e.g., the pixel under investigation 
must reflect > 45% of the incident near-infrared energy. 
When certain rules are applied, various operations may 
take place such as adding a newly derived derivative 
fact to the database or firing another rule. Rules can be 
implicit (slope is high) or explicit (e.g., slope > 70%). It 
is possible to chain together rules, e.g., IF c THEN d; 
IF d THEN e; therefore IF c THEN e. It is also possi- 
ble to attach confidences (e.g., 80% confident) to facts 
and rules. For example, a typical rule used by the MY- 
CIN expert system is (Darlington, 1996): 


IF the stain of the organism is gram-negative 
AND the morphology of the organism is rod 
AND the aerobicity of the organism is anaerobic 
THEN there is strong suggestive evidence (0.8) that the 
class of the organism is Enterobacteriaceae. 


Following the same format, a typical remote sensing 
rule might be: 
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TABLE 10-1 Characteristics that distinguish knowledge-based expert systems from conventional algorithmic problem-solving 


systems (based on Darlington, 1996). 


Characteristics 


Knowledge-Based Expert System 


Conventional Algorithmic System 


Paradigm 


Heuristic. Based on “rules of thumb”. The solution steps are 
implicit (not determined by the programmer). The “solution” 
is not always correct. It is a declarative problem-solving 
paradigm. 


Algorithmic. Solution steps are explicitly 
written by the programmer. The correct 
answer is usually obtained. It is a 
procedural problem-solving paradigm. 


Method of Operation 


Reasons with symbols. Infers the conclusions from known 
premises. Inference engine decides the order in which the 
premises are evaluated. 


Computation. Processes numeric data by 
sorting, calculating, and computing to 
obtain an answer. 


Processing Unit 


Knowledge. Usually represented in the form of rules and 
conditions. Knowledge is active in that an expert system 


Data. Usually represented in arrays or 


records in languages like C**. Data are 


knowledge. 


can reason with the knowledge provided to infer new 


typically passive in that they do not give 
rise to further generations of data. 


Control Mechanism 
knowledge. 


Inference engine is usually separate from the domain 


Data or information and control are usually 
integrated. 


Fundamental Components 


Expert system = inference + knowledge. 


Conventional algorithmic system = 
algorithm(s) + data. 


Explanation Capability 


Yes. An explicit trace of the chain of steps underlying the No. 
reasoning processes allows the user to find out how the 
expert system arrived at its conclusion or why the system is 
asking for an answer to a particular question. 


IF blue reflectanceis(Condition) <15% 
AND green reflectance is (Condition) < 25% 
AND red reflectance is (Condition) < 15% 
AND near-infrared reflectance is (Condition)> 45% 
THEN there is strong suggestive evidence (0.8) that the 
pixel is vegetated. 


Decision Trees The best way to conceptualize an ex- 
pert system is to use a decision-tree structure where 
rules and conditions are evaluated in order to test hy- 
potheses (Figure 10-3). When decision trees are orga- 
nized with hypotheses, rules, and conditions, each 
hypothesis may be thought of as the trunk of a tree, 
each rule a limb of a tree, and each condition a leaf. 
This is commonly referred to as a hierarchical decision- 
tree classifier (e.g., Swain and Hauska, 1977; Jensen, 
1978; DeFries and Chan, 2000; Stow et al., 2003; 
Zhang and Wang, 2003; Friedl et al., 2010). The pur- 
pose of using a hierarchical structure for labeling ob- 
jects is to gain a more comprehensive understanding of 
relationships among objects at different scales of ob- 
servation or at different levels of detail (Tso and 
Mather, 2001). 


A decision tree takes as input an object or situation de- 
scribed by a set of attributes and returns a decision. 
The input attributes can be discrete or continuous. The 
output value can also be discrete or continuous. Learn- 
ing a discrete-valued function is called classification 
learning. However, many applications require the ex- 
traction of biophysical information (e.g., LAI) about a 
pixel or patch of pixels. Such classification is based on 


the use of continuous functions and is called regression 
learning (Lawrence and Wright, 2001; Jensen et al., 
2009; Im et al., 2012a). We will concentrate on Boolean 
classification wherein each example is classified as true 
(positive) or false (negative). A decision tree reaches its 
decision by performing a sequence of tests (Russell and 
Norvig, 2010; Im et al., 2012b). 


Decision trees predict class membership by recursively 
partitioning a dataset into more homogeneous subsets 
(DeFries and Chan, 2000; Im et al., 2012ab). We will 
first use a simple hypothetical GIS-related example to 
demonstrate the logic that should be adopted when 
building the knowledge domain. 


Identify the hypothesis: The expert in charge of creating 
the knowledge domain identifies a hypothesis (prob- 
lem) to be addressed. This may be a formal hypothesis 
to be tested using inductive logic and confidence levels 
or an informal hypothesis that is in search of a logical 
conclusion: 


¢ Hypothesis 1: the terrain (pixel) is suitable for resi- 
dential development that makes maximum use of 
solar energy (i.e., I will be able to put solar panels on 
my roof). 


Specify the expert system rules: Heuristic rules that the 
expert has learned over time are the heart and soul of 
an expert system. If the expert’s heuristic rules of 
thumb are indeed based on correct principles, then the 
expert system will most likely function properly. If the 
expert does not understand all the subtle nuances of 
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Building Blocks of an Expert System 


Hypothesis 


Hypothesis 


Rules 


Conditions 


FIGURE 10-3 The fundamental building blocks of an expert system include hypotheses (problems), rules, and conditions. 
The rules and conditions operate on data (information). It is possible to address more than one hypothesis using an expert 


system. 


the problem, has left out important variables or inter- 
action among variables, or applied too much signifi- 
cance (weight) to certain variables, the expert system 
outcome may not be accurate. Therefore, the creation 
of accurate, definitive rules is extremely important 
(Hodgson et al., 2003). Each rule provides the specific 
conditions to accept the hypothesis to which it belongs. 
For example, a single rule that might be associated 
with hypothesis | is: 


* specific combinations of terrain slope, aspect, and 
proximity to shadows result in maximum exposure 
to sunlight. 


Specify the rule conditions: The expert would then spec- 
ify one or more conditions that must be met for each 
rule. For example, conditions for the rule stated above 
might include: 


« slope > 0°, AND 


* slope < 10° (ie., the terrain should ideally lie on 
terrain with 1° to 9° slope), AND 


* aspect > 135°, AND 


* aspect < 220° (i.e., in the Northern Hemisphere the 
terrain should ideally face south between 136° and 
219° to obtain maximum exposure to sunlight), 
AND 


* the terrain is not intersected by shadows cast by 
neighboring terrain, trees, or other buildings 
(derived from a viewshed model). 


In this case, the hierarchical decision-tree diagram 
would look like Figure 10-4. 


To function properly, the expert system must have ac- 
cess to the data (variables) required by the rules. In this 
case three types of spatial information would be re- 
quired for one rule: 1) slope, 2) aspect, and 3) a shadow 
viewshed file derived from a detailed digital elevation 
model. 


Inference Engine 


The terms reasoning and inference are generally used to 
describe any process by which conclusions are reached 
(Russell and Norvig, 2010). Thus, the hypotheses, 
rules, and conditions are passed to the inference engine 
where the expert system is implemented. One or more 
conditional statements within each rule are evaluated 
using the spatial data (e.g., 135° < aspect < 220°). Mul- 
tiple conditions within a rule are evaluated based on 
Boolean AND logic. While all of the conditions within 
a rule must be met to satisfy the rule, any single rule 
within a hypothesis can cause that hypothesis to be ac- 
cepted or rejected. In some cases, rules within a hy- 
pothesis disagree on the outcome and a decision must 
be made using rule confidences (e.g., a confidence of 
0.8 in a preferred rule and a confidence of 0.7 in anoth- 
er) or the order of the rules (e.g., preference given to 
the first rule) as the factor. The confidences and order 
associated with the rules are normally stipulated by the 
expert. 


The inference engine interprets the rules in the knowl- 
edge base to draw conclusions. The inference engine 
may use backward- and/or forward-chaining strategies. 
Both backward and forward inference processes con- 
sist of a chain of steps that can be traced by the expert 
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FIGURE 10-4 A human-derived decision-tree expert system with a rule and conditions to be investigated by an inference en- 


gine to test hypothesis 1. 


system. This enables expert systems to explain their 
reasoning processes, which is an important and posi- 
tive characteristic of expert systems. You would expect 
a doctor to explain how he or she came to a certain di- 
agnosis regarding your health. An expert system can 
provide explicit information about how a particular 
conclusion (diagnosis) was reached. 


An expert system shell provides a customizable infer- 
ence engine. Expert system shells come equipped with 
an inference mechanism (backward chaining, forward 
chaining, or both) and require knowledge to be entered 
according to a specified format. Expert system shells 
qualify as languages, although certainly with a narrow- 
er range of application than most programming lan- 
guages (PC AI, 2002). Typical artificial intelligence 
programming languages include LISP, developed in 
the 1950s, PROLOG, developed in the 1970s, and now 
object-oriented languages such as C**. 


On-Line Databases 


The rules and conditions may be applied and evaluated 
using data and/or information stored in on-line data- 
bases. The databases can take a variety of forms. It can 
be spatial and consist of remotely sensed images and 
thematic maps in raster and vector format. However, 
the database may also consist of charts, graphs, algo- 
rithms, pictures, and text that are considered important 
by the expert. The database should contain detailed, 
standardized metadata. 


Expert Systems Applied to Remote 
Sensor Data 


The use of expert systems in remote sensing research 
will be demonstrated using two different methodolo- 
gies used to create the rules and conditions in the 
knowledge base. The first expert system classification is 
based on the use of formal rules developed by a human 
expert. The second example involves expert system 


rules not generated by humans, but derived automati- 
cally by an inductive machine-learning algorithm 
based on training data that is input by humans into the 
system. Both methods will be used to identify white fir 
forest stands on Maple Mountain in Utah County, UT, 
using Landsat Enhanced Thematic Mapper Plus 
(ETM”*) imagery and several topographic variables ex- 
tracted from a digital elevation model of the area. The 
comparison will demonstrate important characteristics 
of human expert versus machine-learning rule develop- 
ment strategies. 


Decision-Tree Classification 
Based on Human-Derived 
Rules 


This example is based on rules specified by a human 
expert to map white fir (Abies concolor) forest on Ma- 
ple Mountain located in Utah County, UT. Maple 
Mountain rises from the 5,000 ft. valley floor to 10,200 
ft. above sea level (ASL). A Landsat ETM” panchro- 
matic image of the mountain and valley obtained on 
August 10, 1999, is shown in Figure 10-5a. A color- 
composite of bands 4, 3, and 2 (NIR, red, green) is 
shown in Figure 10-5b. A Normalized Difference Vege- 
tation Index (NDVI) image derived from the Landsat 
ETM” red and near-infrared bands (4 and 2) is dis- 
played in Figure 10-5c. 


The goal of this exercise is to accurately extract the 
spatial distribution of forest land-cover information 
using the original remotely sensed data, image-derived 
products (e.g., NDVI), elevation from a digital eleva- 
tion model (DEM), and DEM-derived products (e.g., 
slope and aspect). A U.S. Geological Survey 30 x 30 m 
digital elevation model is shown in Figure 10-5d. Con- 
tour, shaded relief, slope, and aspect were extracted 
from the digital elevation model (Figures 10-Se 
through 1). 
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TABLE 10-2 A hypothesis (class), variables, and conditions necessary to extract white fir (Abies concolor) forest cover infor- 
mation from Maple Mountain, UT, using remote sensing and digital elevation model data. The Boolean logic with which these 
variables and conditions are organized within a chain of inference may be controlled by the use of rules and sub-hypotheses. 


Hypothesis Variables Conditions 
White fir (Abies concolor) Aspect Aspect = 300° to 45° 
Elevation Elevation >1,200 m 
Slope ° fe} 
I = 25mtoro 
Multispectral Rouen ae 


Remote sensing reflectance 
TM band 1 Blue = 44 to 52 
TM band 2 Green = 31 to 40 
TM band 3 Red = 22 to 32 
TM band 4 Near-infrared = 30 to 86 
TM band 5 Mid-infrared = 19 to 47 
NDVI = 0.2 to 0.7 


Hypotheses to Be Tested 


The hypothesis (class) to be tested (extracted) from the 
spatial data was white fir (Abies concolor). Many other 
types of land cover are present in the scene but we will 
focus our attention on white fir. The structure of the 
expert system logic is displayed in decision-tree format 
in the expert system interface shown in Figure 10-6. 
Note that the hypothesis represents the base of the de- 
cision tree (laid on its side). 


Rules (Variables) 


A human expert who worked for the Uinta National 
Forest developed the knowledge base (hypotheses, 
rules, and conditions) to identify white fir habitat from 
all other land cover in this area. The rules and condi- 
tions were based on remote sensing multispectral re- 
flectance characteristics and derivatives (e.g., NDVI), 
elevation above sea level, and the expert’s knowledge of 
microclimate and soil moisture conditions that are 
controlled primarily by terrain slope and aspect (Table 
10-2). 


Conditions 


The expert identified very specific conditions that are 
associated with the remote sensing reflectance data, el- 
evation, slope and aspect. This part of Utah is in a 
semiarid mountain and range province and receives 
relatively little moisture in the summer months. There- 
fore, south-facing slopes usually have reduced soil 
moisture conditions throughout much of the summer. 
The expert knows that white fir requires substantial 
year-round soil moisture. Therefore, it thrives best on 
north-facing slopes (300° < aspect < 45°). White fir fa- 
vors elevations > 1,900 m ASL. It out-competes other 
forest types when the slope is between 25° and 50°. 


White fir forest physiology causes it to have relatively 
distinct red and near-infrared reflectance characteris- 
tics when compared with other vegetation types on the 


mountain. This translates into useful normalized dif- 
ference vegetation index (NDVI) values, 


Prua~ PrM3 


NDVI = (10.1) 


Prva t Pru3 


based on Landsat 7 ETM™ band 3 (red) and band 4 
(near-infrared) reflectance data. The specific multi- 
spectral remote sensing conditions for each band and 
for the NDVI are summarized in Table 10-2. The hy- 
pothesis, rules, and conditions specified in the expert 
system interface are summarized in Figure 10-6. 


Inference Engine 


The hypothesis, rules, and conditions created by the ex- 
pert were passed to the inference engine. The inference 
engine processed the rules and conditions in conjunc- 
tion with the required spatial data. This produced the 
classification map depicting the spatial distribution of 
white fir displayed in planimetric map format in Figure 
10-7e and draped over the Landsat ETM* color com- 
posite and digital elevation model in Figure 10-7f. 


Unfortunately, this classification took quite a bit of re- 
search on the part of the human expert to come up 
with the expert system rules and conditions. It would 
be much better if there were some way for the expert 
system to generate its own rules based on training da- 
ta. This leads us to a discussion of machine learning. 


Classification Based on 
Machine Learning Decision 
Trees and Regression Trees 


The heart of an expert system is its knowledge base. 
The usual method of acquiring knowledge in a com- 
puter-usable format to build a knowledge base involves 
human domain experts and knowledge engineers, as 
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Maple Mountain in Utah County, UT 


4 4 R , ” | 


a. Landsat ETM+ panchromatic —_b. Landsat ETM* color-composite c. NDVI extracted using 
image obtained on August 10, 1999. (RGB = bands 4, 3, 2). ETM? bands 3 and 4. 
: ’ ; | q 


d. USGS 30 x 30m e. Contours extracted from the f. Contours overlaid on color-coded 
digital elevation model. digital elevation model. digital elevation model. 


g. Shaded-relief representation h. Slope derived from the 1. Aspect derived from the 
of the digital elevation model. digital elevation model. digital elevation model. 


FIGURE 10-5 a-i) Datasets used to identify white fir (Abies concolor) on Maple Mountain in Utah County, UT (Landsat ETM* 
data courtesy of NASA and digital elevation model courtesy of USGS). 
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Human-derived Hierarchical Decision-Tree Hypothesis, Rules, and Conditions 
for the Classification of White Fir (Abies concolor) on Maple Mountain, UT 
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FIGURE 10-6 Hierarchical decision tree associated with a knowledge-based expert system classification of white fir (Abies 
concolor) on Maple Mountain, UT. A human expert identified the hypotheses, rules, and conditions. 


previously discussed. The human domain expert ex- 
plicitly expresses his or her knowledge about a subject 
in a language that can be understood by the knowledge 
engineer. The knowledge engineer translates the do- 
main knowledge into a computer-usable format and 
stores it in the knowledge base. 


This process presents a well-known problem in creating 
expert systems that is often referred to as the knowl- 
edge acquisition bottleneck (Jensen et al., 2009). The 
reasons are: 


¢ the process requires the engagement of the domain 
expert and/or knowledge engineer over a long 
period of time, and/or 


¢ although experts are capable of using their knowl- 
edge for decision-making, they are often incapable 
of articulating their knowledge explicitly in a format 
that is sufficiently systematic, correct, and complete 
to be used in a computer application. 


Remote sensing scientists have acknowledged these dif- 
ficulties when building knowledge bases for image 
analysis. For example, operational monitoring of land 
cover from satellite requires automated procedures for 


analyzing large volumes of data through time (Fried et 
al., 2010). To solve such problems, much effort has 
been exerted to automate the building of expert system 
knowledge bases and production rules (e.g., Huang 
and Jensen, 1997; Tso and Mather, 2001; Tullis and 
Jensen, 2003; Jensen et al., 2009; Im et al., 2012ab). 


Machine Learning 


Machine learning is defined as the science of computer 
modeling of learning processes. It enables a computer 
to acquire knowledge from existing data or theories us- 
ing certain inference strategies such as induction or de- 
duction. We will focus only on inductive learning and 
its application in building knowledge bases for image 
analysis expert systems. 


A human being has the ability to make accurate gener- 
alizations from a few scattered facts provided by a 
teacher or the environment using inductive inferences. 
This is called inductive learning (Huang and Jensen, 
1997). In machine learning, the process of inductive 
learning can be viewed as a heuristic search through a 
space of symbolic descriptions for plausible general de- 
scriptions, or concepts, that explain the input training 
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a. Landsat ETM™ (RGB = bands 4, 3, 2) image obtained b. Landsat ETMt (RGB = bands 5, 4, 2) image draped 


on August 10, 1999, draped over a digital elevation model. over a digital elevation model (azimuth = 90°). 
——. 


c. Terrestrial photograph of Maple Mountain (azimuth = 113°). d. Landsat ETM+ (RGB = bands 4, 3, 2) image draped 
over a DEM viewed from the same azimuth as in part c. 


f, Expert system classification of white fir draped 
over the color composite and digital elevation model. 


Expert System Classification 
of White Fir (Abies concolor) on 
Maple Mountain in Utah County, UT 


e. Expert system classification of white fir. 


FIGURE 10-7 Expert system classification of white fir (Abies concolor) on Maple Mountain in Utah County, UT. ab) Landsat 
ETM* color composites draped over a digital elevation model. c) Terrestrial natural color photograph. d) Maple Mountain 
viewed from an azimuth of 113°. e,f) Vertical and terrestrial views of the white fir classification derived using human-specified 
rules and an expert system. 
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data and are useful for predicting new data. Inductive 
learning can be formulated using the following symbol- 
ic formulas (Michalski, 1983): 

Viel 


(E,>D,) (10.2) 


Vi,jel (E,>~D,), if(j #7) (10.3) 
where D; is a symbolic description of class i, E; is a 
predicate that is true only for the training events of 
class i, [is a set of class names, ~ stands for “negation,” 
and => stands for “implication.” Expression 10.2 is 
called the completeness condition and states that every 
training event of some class must satisfy the induced 
description D; of the same class. However, the opposite 
does not have to hold because D; is equivalent to or 
more general than £;. This means that D; may include 
some features that do not exist in some examples in £;. 
Expression 10.3 is called the consistency condition and 
states that if an event satisfies a description of some 
class, it cannot be a member of a training set of any 
other class. The task of inductive learning is to find 
through the space of descriptions the general descrip- 
tion set D = {Dj, Do, ..., Dj} for the class set K = (Ky, 
K), ..., K;) that satisfies the completeness condition and 
also, in some cases, the consistency condition (Huang 
and Jensen, 1997). 


The general description set, or concept, D, resulting 
from inductive learning can be represented by a variety 
of formalisms, including production rules. This means 
that inductive learning can be used to build knowledge 
bases for expert systems because production rules are 
the most popular form of knowledge representation in 
expert systems. A motivation for the use of this ap- 
proach to build a knowledge base is that it requires 
only a few good examples to function as training data. 
This is often much easier than explicitly extracting 
complete general theories from the domain expert. 


It is important to remember that machine-learning de- 
cision trees can generate categorical output informa- 
tion (e.g., forest, agriculture, urban land cover) (e.g., 
Xian et al., 2009), while regression trees can predict 
continuous quantitative output such as leaf-area-index 
(LAI) or percent canopy cover (Clark Labs CTA, 2014; 
Xian et al., 2011; Im et al., 2012b). 


There are a number of machine-learning, data-mining 
decision tree/regression tree programs that can be used 
to develop the production rules, including: 


¢ Salford Systems (2014a), Inc. CART® Classifica- 
tion and Regression Trees. CART is based on math- 
ematical theory introduced in 1984 by four 
statisticians at Stanford University and the Univer- 


sity of California at Berkeley. Salford Systems’ 
implementation of CART is the only decision-tree 
software embodying the original proprietary code. 
The CART creators continue to collaborate with 
Salford Systems to provide CART with proprietary 
advances. CART has been shown to be very effective 
for land-cover classification (e.g., Sexton et al., 
2013). 


¢ TIBCO S-Plus (2014) includes >4,200 data model- 
ing and statistical analysis functions including 
extensive decision-tree/regression-tree creation and 
analysis. 


¢ R Development Core Team (2014) provides the R 
language and environment for statistical computing 
and graphics. Tree models are available in R via the 
user-contributed packages rpart and tree (Therneau 
et al., 2014). 


¢ RuleQuest Research, Inc. (2014) C5.0®@ (Unix, 
Linux), See5® (for Windows) and Cubist® devel- 
oped by J. Ross Quinlan. C4.5 and C5.0 have been 
used extensively for land-cover classification and for 
predicting continuous variables such as leaf-area- 
index (e.g., Im and Jensen, 2005; Sexton et al., 2013; 
Im et al., 2012ab). IDRISI’s Classification Tree 
Analysis (CTA) uses C4.5 (Clark Labs, CTA, 2014). 
The NASA global MODIS Collection 5 Land 
Cover type product (MCD12Q1) is based on the use 
of C4.5 (Friedl et al., 2010). 


CART, S-Plus, C4.5, C5.0, Cubist, and R were not de- 
veloped specifically for remote sensing digital image 
processing. Rather, they were developed for a diverse 
array of intensive data mining applications in such 
fields as pharmaceutical and biomedical data analysis, 
population studies, medical image analysis, and other 
data-rich applications. Fortunately, the decision rules 
derived using these machine-learning programs are 
routinely used to build a decision- and/or regression- 
tree classifier within remote sensing digital image pro- 
cessing programs such as ENVI’s Interactive Decision 
Tree tool (Exelis ENVI, 2014), ERDAS Imagine’s 
Knowledge Engineer (Hexagon ERDAS Imagine, 
2014), or Clark Labs Classification Tree Analysis 
(Clark Labs CTA, 2014). 


The C5.0 program will be used to demonstrate the use 
of machine-learning decision trees to create production 
rules. Most decision-tree programs have the following 
advantages: (Quinlan, 2003; Lu and Weng, 2007; Clark 
Labs CTA, 2014; Jensen et al., 2009; Exelis ENVI, 
2014; RuleQuest, 2014): 


¢ To maximize interpretability, its classifiers are 
expressed as decision trees or sets of if—then rules, 
forms that are generally easier to understand than 
neural networks. 
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¢ The knowledge learned can be stored in a produc- 
tion rule format that can be used to create a knowl- 
edge base for a rule-based expert system. 


¢ It is flexible. Unlike many statistical approaches, it 
does not depend on assumptions about the distribu- 
tion of attribute values or the independence of the 
attributes themselves. This is very important when 
incorporating ancillary GIS data with remotely 
sensed data because they usually have different attri- 
bute value distributions, and some of the attributes 
may be correlated. 


* Itis based on a decision-tree learning algorithm that 
is one of the most efficient forms of inductive learn- 
ing. The time taken to build a decision tree increases 
only linearly with the size of the problem. 


¢ Weights can be applied to different classes and 
boosting may take place, which reduces bias during 
supervised learning. 


The procedure of applying the inductive learning tech- 
nique to automatically build a knowledge base for a re- 
mote sensing image analysis expert system that 
incorporates GIS data involves training, decision-tree 
generation, and the creation of production rules. The re- 
sultant production rules compose the knowledge base 
and can be used by an expert system inference engine 
to perform the final image classification, e.g., using 
ENVYI’s interactive decision-tree tool or IDRISI’s clas- 
sification tree analysis. 


Decision-Tree Training 


A suitable classification system and a sufficient number 
of training samples are prerequisites for a successful 
training (Lu and Weng, 2007).The objective of training 
is to provide examples of the concepts to be learned. 
When building a knowledge base for image classifica- 
tion, the examples should be a set of training objects, 
each of which is represented by an attribute value class 
vector such as: 


[attribute_l, ..., attribute_n, class_i] 


Attributes are then collected for each class of informa- 
tion (e.g., spectral reflectance values in various bands, 
elevation, aspect, slope). Stratified random sampling 
may be the most appropriate way to locate the candi- 
date training class pixels because it guarantees that a 
minimum number of samples are selected from each 
class (strata) (Congalton and Green, 2009). The learn- 
ing algorithm attempts to extract from this attribute 
training dataset some generalized concepts, i.e., rules 
that can be used to classify the remaining data (Im et 
al., 2012b). 
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Decision-Tree Generation 


The CS5.0 learning algorithm first generates decision 
trees from the training data. The decision trees are then 
transformed into production rules. A raw decision tree 
(prior to the formation of production rules) can be 
viewed as a classifier composed of leaves that corre- 
spond to classes, decision nodes that correspond to at- 
tributes of the data being classified, and arcs that 
correspond to alternative values for these attributes. A 
hypothetical example of a decision tree is shown in 
Figures 10-8a and b. 


A recursive “divide and conquer” strategy is used by 
C5.0 to generate a decision tree from a set of training 
data. The training dataset S' is divided into subsets S;, 
..., 5, according to a), ..., d,, which are the possible 
values of a single attribute A. This generates a decision 
tree with A being the root and Sy), ..., S,, corresponding 
to subtrees 77, ..., J), (Figure 10-8c). 


The stop condition for such a procedure will eventually 
be satisfied, resulting in a final decision tree. The goal 
is to build a decision tree that is as small as possible. 
This ensures that the decision-making by the tree is ef- 
ficient. The goal is realized by selecting the most infor- 
mative attribute at each node so that it has the power 
to divide the dataset corresponding to the node into 
subsets as pure (homogeneous) as possible (DeFries 
and Chan, 2000). C5.0’s attribute selection criteria are 
based on the entropy measure from communication 
theory where, at each node, the attribute with the mini- 
mum entropy is selected to divide the dataset. 


From Decision Trees to Production 
Rules 


Decision trees are often too complex to be understood, 
especially when they are large. A decision tree is also 
difficult to maintain and update. Therefore, it is desir- 
able to transform a decision tree to another representa- 
tion of knowledge that is adopted commonly in expert 
systems, such as production rules. 


A production rule can be expressed in the following 
general form (Jackson, 1990): 


Pisce 


—> Q),.--; Qn (10.4) 


with the meaning 


* if premises (or conditions) P, and ... P,,, are true, 
* then perform actions Q; and ... Q,,. 
In fact, each path from the root node to a leaf in a deci- 


sion tree can be translated into a production rule. For 
example, the path from the root node to the left most 
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Example of a Decision Tree 


Node (attribute) 
Arc 


(a single value, ——————_> 


a group of values, 
or a range of values 
of the attribute) 


Best 
Leaf (class) 


Wetland Dead 


Dead Bare 
veg. veg. soil 


e.g., red 


Band 1 > 82 

/ Soil = best: wetland 

/ Soil = good: dead vegetation 
/ Soil = fair: dead vegetation 
/ Soil = poor: bare soil 

Band 1 < 82 

<40 / Band 2 > 40: wetland 

- / Band 2 < 40: water 


e.g., Near-infrared 


b. 


Wetland Water 


FIGURE 10-8 Example of a decision tree. a) A dataset consisting of three attributes (band 1, band 2, and soils) is classified 
into one of four classes: water, wetland, dead vegetation, and bare soil. b) Another method of presenting the decision tree 
shown in (a). c) A decision tree generated by dividing dataset S into subsets (based on Huang and Jensen, 1997). 


leaf in the decision tree in Figure 10-8a can be repre- 
sented by a production rule such as 


(band | > 82), (soil= poor) —> (class = bare soil). 


Several problems must be solved when transforming a 
decision tree into production rules. First, individual 
rules transformed from the decision tree may contain 
irrelevant conditions. C5.0 uses a pessimistic estimate 
of the accuracy of a rule to decide whether a condition 
is irrelevant and should be deleted. Second, the rules 
may cease to be mutually exclusive and collectively ex- 
haustive. Some rules may be duplicative or may con- 
flict. This is a common problem for rule-base building 
using either a manual or automated approach. Usually, 
a rule-based system has some conflict resolution mech- 
anism to deal with this problem. The approach adopt- 
ed by C5.0 includes the use of rule confidence voting 
and a default class. In other words, each rule is as- 
signed a confidence level based on false-positive errors 
(the number of training objects that were incorrectly 
classified as class C by a rule). If an object can be as- 
signed to more than one class by two or more rules, a 
summary vote of all rule confidences concerned is tab- 
ulated and the class with the highest number of votes 
wins (Quinlan, 2014). 


Unfortunately, some objects to be classified may satisfy 
no rules or they may satisfy conflicting rules with equal 


confidence levels. When this occurs, a default class 
might be used. There are a number of ways a default 
class can be identified. One approach is to have the de- 
fault class be the one that contains the most training 
objects not satisfying any rule. 


The quality of the resultant rules can be evaluated by 
predicting error rates derived by applying the rules on a 
test dataset. Because the rules are easy to understand, 
they can also be examined by human experts. With 
caution, they may be edited directly. 


Case Study 

To demonstrate the utility of machine learning for the 
creation of production rules, the C5.0 algorithm was 
trained with the Maple Mountain data previously dis- 
cussed. This resulted in the creation of the rules sum- 
marized in Figure 10-9a. The classification map 
produced from the machine-learning rules is shown in 
Figure 10-9b. A per-pixel confidence map of the deci- 
sion-tree classification is shown in Figure 10-9c. 


Advantages of Decision-Tree Classifiers 

Expert system decision-tree classifiers have several 
characteristics that make them attractive when com- 
pared with other classification methods: 


¢ Machine-learning algorithms may be used to train 
expert systems by creating rules and conditions 
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Machine-learning Hierarchical Decision-Tree Classification 
of White Fir (Abies concolor) on Maple Mountain, UT 


Rules Conditions 


=> Rude A }-______________ 9 Elevation <= 2088 


es 2 TM band? > 52! 


exe Pade B | 2 ™ band 4 <= 82| 


Hypotheses 
Benson e305) 
2 TM band > 48) 

wae) Bender ea] 
2 TM band 5<=47 

= Fide D | 9. TM band 5 > 47} 
‘2M band 4 <o97| 
2 TM bend 5 > 47/ 


= Rue E | ? Aspect > 05 


? Elevation > 2498 
Bh Witte Fee (ties concob) ? Skge> 4! 


2 TM band 3c» 23 


es Pade F 2 TM band 4> 98) 


—Irinomanedintad 


= Rude G -———______. 9 IM band 5 <= 47 


a. Expert system rules for classifying white fir (Abies concolor) derived using machine learning. 


Confidence f) (ail 


b. Classification of white fir (Abies concolor) on Maple c. Confidence map. 
Mountain based on rules derived using machine 
learning draped over a three-dimensional view 
of Landsat ETM+ imagery. 


FIGURE 10-9 a) Hierarchical decision-tree classification of white fir (Abies concolor) on Maple Mountain in Utah County, UT. 
The C5.0 machine-learning decision tree developed these production rules based on the characteristics of the training data. 
b) Results of applying the production rules. c) A confidence map of the classification. 
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directly from the training data without human inter- 
vention (e.g., Huang and Jensen, 1997; Im et al., 
2012ab; RuleQuest, 2014; Salford Systems, 2014a). 
This is a very important advancement because the 
expert system can adapt when new learning data are 
provided. Otherwise, an expert system cannot learn 
by example. 


¢« You can evaluate the output of the expert system 
(e.g., whether a certain pixel is ideal or not for resi- 
dential development) and work backward to iden- 
tify how a conclusion was reached. This is in 
contrast to neural networks where the exact nature 
of the decision process is often lost in the weights 
used in the hidden layers (Qiu and Jensen, 2004). 


¢ Until recently, maximum likelihood classification 
was the most common method used for supervised 
classification of remote sensor data (McIver and 
Friedl, 2002; Lu and Weng, 2007). This method 
assumes that the probability distributions for the 
input classes possess a multivariate normal distribu- 
tion. Increasingly, nonparametric classification 
algorithms are being used such as decision trees, 
which make no assumptions regarding the distribu- 
tion of the data (Fried1 et al., 2010). 


¢ The decision tree can reveal nonlinear and hierar- 
chical relationships among the input variables and 
use them to predict class membership. 


A large body of evidence demonstrates the ability of 
machine-learning techniques (particularly decision 
trees and neural networks) to deal effectively with tasks 
that involve multi-dimensional data (e.g., hyperspec- 
tral). Reducing a dataset to just two or three variables 
(e.g., bands) is becoming an outdated notion (Gahe- 
gan, 2003). The new thinking 1s to let the geographical 
data themselves “have a stronger voice” rather than let 
statistics derived from the dataset dictate the analysis 
(e.g., the means and covariance matrices used in max1- 
mum likelihood classification). 


tae Random Forest Classifier 


Now that we are comfortable with single decision-tree 
classifiers, it is important to note that there are also 
machine-learning algorithms that make use of multiple 
decision trees (i.e., an ensemble) at one time during the 
classification process. Ensemble learning algorithms 
(eg., Random Forests, bagging, boosting) have 
emerged as potentially more accurate and robust alter- 
natives to conventional parametric (e.g., maximum 
likelihood) or other machine-learning algorithms (e.g., 
individual decision trees, neural networks) (Rodriguez- 
Galiano et al., 2012ab; Chen et al., 2014). 


Random Forests grows many classification decision 
trees. To assign an unclassified pixel (or OBIA object) 
with its associated attributes (e.g., spectral reflectance, 
elevation, slope) into a class, the unclassified input pix- 
el values are run through each of x decision trees in the 
forest (e.g., 200 trees). Each decision tree classifies the 
pixel into one of y classes, i.e. it “votes” for that class. 
The forest assigns the pixel to the class having the most 
votes from all the trees in the forest (Breiman, 2001; 
Breiman and Cutler, 2014; Hayes et al., 2014). 


Each of the individual decision trees is grown as fol- 
lows (Breiman and Cutler, 2014): 


1. If the number of cases in the training set is N, 
sample N cases at random, but with replacement, 
from the original data. This sample will be the 
training set for growing the tree. Generally, for 
every tree grown, only about 70% of the training 
data is used and about 30% of the training data are 
left out-of-bag (OOB), i.e., they are not used. 

2. If there are M input variables, a number m <M is 
specified such that at each node, m variables are 
selected at random out of the and the best split 
of these m is used to split the node. The value of m 
is held constant as the forest of trees is grown. It is 
suggested that m be the square root of M. 


3. Each decision tree is grown to the largest extent 
possible. There is no pruning. 


Individual classes can be weighted a priori if desired. 
Random Forests runs efficiently on very large datasets, 
which is a useful characteristic when analyzing remote 
sensor data. It also provides estimates of what variables 
are most important in the classification. 


Numerous studies have evaluated Random Forest clas- 
sification with other types of image classification (e.g., 
Pal, 2005; Guo et al., 2011; Rodriguez-Galiano et al., 
2012ab). Random Forest freeware is available in R via 
the user-contributed package randomForest (R Devel- 
opment Core Team, 2014; Liaw and Weiner, 2014). 
Random Forest commercial software is available 
through Salford Systems (2014b). 


ae Support Vector Machines 


Support vector machines (SVMs) are another type of 
machine-learning software that can be used for remote 
sensing classification (Pal and Mather, 2005; Jensen et 
al., 2009; Duro et al., 2012). Comparative studies of 
SVMs versus decision trees and artificial neural net- 
works have demonstrated the robustness of SVMs, es- 
pecially for small training sets (e.g., Huang et al., 2002; 
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Support Vector Machine Classification (Linear) 
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FIGURE 10-10 A support vector machine classification re- 
lies heavily on the training class samples on the edge of the 
margin to identify the location of the separating hyperplane 
(based on Meyer, 2014). 


Foody and Mathur, 2004). SVMs find an optimal sepa- 
rating hyperplane (OSH) between classes using the 
training samples (Foody and Mathur, 2006; Van der 
Linden and Hostert, 2009). However, in this case we 
are looking for the optimal separating hyperplane be- 
tween two classes by maximizing the margin between 
the classes’ closest training samples, as shown in Fig- 
ure 10-10. The points lying on the boundaries are 
called support vectors, and the middle of the margin is 
the optimal separating hyperplane (Meyer, 2014). 
Training data points on the “wrong” side of the dis- 
criminant margin are given negative weights to reduce 
their influence. When it is not possible to find a linear 
separator such as shown in Figure 10-10, the data 
points can be projected into a higher-dimensional 
space (using kernel techniques). A program that per- 
forms all of these tasks is called a support vector ma- 
chine (Meyer, 2014). 


SVMs focus on solving the optimization problem with 
training samples using the following equations (Jensen 
et al., 2009): 


Tr nr 
miny 5 c sie 5 r+C > Gi (10.5) 
i=l 
I 
yiow" @(x)+b)21-G,  (C;20) 


where €, are positive slack variables used to allow 
some of the samples to fall on the wrong side of the hy- 
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perplane, C XC, is a term used to penalize solutions 
for which ¢, are very large, and we -@(x,) +b is a hy- 
perplane in a highly dimensional feature space (Su et 
al., 2006). The basic SVM approach may be extended 
to optimize nonlinear surfaces using the following de- 
cision function (Jensen et al., 2009): 


nr 
y a,y,K(x,x,;) +b 


i=1 


Ax) = (10.6) 


where a; are nonnegative Lagrange multipliers used to 
search the OSH, and K(x, x;) is a kernel function, 
which replaces the inner product (x - x;) in order to 
solve computational problems in a higher dimensional 
space (Wang et al., 2005). The selection of the kernel 
function k is critical for producing successful results. 
Several widely used SVM types of kernels include lin- 
ear, polynomial, Gaussian, sigmoid, and spectral angle 
mapper kernels. 


One advantage of SVMs is that small training samples 
selected on the boundaries of classes, where mixed pix- 
els are common, can lead to accurate classification 
(Foody and Mathur, 2006). Pal (2005) compared the 
accuracies of two supervised classification algorithms 
using Landsat ETM™ data: Random Forests and 
SVMs and found that they performed equally well 
(Duro et al., 2012). SVM freeware is available in R via 
the user-contributed package e/077 (R Development 
Core Team, 2014; Meyer, 2013, 2014). 


ae Neural Networks 


A neuron is a cell in the brain whose principal function 
is the collection, processing, and dissemination of elec- 
trical signals (Russell and Norvig, 2010). Neural net- 
works simulate the thinking process of human beings, 
whose brains are a large number of small and intercon- 
nected processors or neurons that work together to 
solve difficult classification and optimization problems 
(Khorram et al., 2011). A neural network reaches a so- 
lution not via a step-by-step algorithm or a complex 
logical program, but in a non-algorithmic, unstruc- 
tured fashion based on the adjustment of the weights 
connecting the neurons in the network (Filippi et al., 
2010). Neural networks have been used to classify vari- 
ous types of remote sensor data and have in certain in- 
stances produced results superior to those of 
traditional statistical classification methods (eg., 
Foody, 1996; Jensen et al., 1999; Ji, 2000; Aitkenhead 
and Aalders, 2011; Santi et al., 2014). This success can 
be attributed to two of the important advantages of 
neural networks: 1) freedom from normal distribution 
requirements (Hu and Weng, 2009; Khorram et al., 
2011), and 2) the ability to adaptively simulate complex 
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Typical Artificial Neural Network Components 


Hidden 
layer(s) 


Input 
layer 


Output 


weights, layer 


WwW. 
L 


Neuron C 


Oo, 
22 
Kon oy 
7) 


Thematic 
map classes 


Spectral reflectance, 
texture, elevation, 
slope, aspect, etc. 


and nonlinear patterns given proper topological struc- 
tures (Filippi and Jensen, 2006). 


Components and Characteristics of a 
Typical Artificial Neural Network 
Used to Extract Information from 
Remotely Sensed Data 


The topological structure of a typical back-propaga- 
tion neural network is shown in Figure 10-11. The arti- 
ficial neural network normally contains neurons 
arranged in three types of layer: 


* an input layer, 
¢ a hidden layer(s), and 
* an output layer. 


The neurons in the input layer might be the multispec- 
tral reflectance values associated with individual pixels 
in the imagery and/or objects if object-based image 
analysis is (OBIA) is used. However, one of the main 
advantages of neural networks is the fact that multiple 
data sources, including non-spectral data, can be used 
as input such as elevation, aspect, slope, etc. (Brown et 
al., 2008). 


The use of neurons in the hidden layer(s) enables the 
simulation of nonlinear patterns in the input data. The 
choice of how many hidden layers and the number of 
nodes per hidden layer is an ongoing basic research 
question (Brown et al., 2008; Hu and Weng, 2009). Ait- 
kenhead and Aalders (2011) suggest that each hidden 


FIGURE 10-11 Components of a typical back-propagation arti- 
ficial neural network (ANN). The network consists of input, hid- 
den, and output layers. The input layer may contain information 
about individual training pixels including spectral reflectance in 
various bands and ancillary data such as elevation, slope, etc. 
Each layer consists of nodes that are interconnected. This inter- 
connectedness allows information to flow in multiple directions 
(i.e., back-propagation can take place) as the network is trained. 
The strength (or weight) of these interconnections is eventually 
learned by the neural network and stored. These weights are 
used during the testing (classification) procedure. The more 
representative the training data, the more likely that the neural 
network will develop weights in the hidden layer that mirror re- 
ality and result in an accurate classification. The output layer 
may represent individual thematic map classes such as water or 
forest (based on Jensen, J. R., Qiu, F and K. Patterson, 2001, 
“A Neural Network Image Interpretation System to Extract 
Rural and Urban Land Use and Land Cover Information from 
Remote Sensor Data,” Geocarto International, 16(1):19-28). 


layer used should have twice as many nodes as the 
number of nodes in the input layer based on Kol- 
mogorov’s theorem for neural network modelling of 
continuous functions. 


Land-cover classes can be mapped using an output lay- 
er that contains a neuron for each class. Or, it is possi- 
ble to use only a single output neuron to predict 
continuous variables such as wheat yield or chlorophyll 
a concentration as demonstrated by Jiang et al. (2004) 
and Gonzalez Vilas et al. (2011). 


Much like a supervised classification, an artificial neu- 
ral network requires training and testing (classifica- 
tion) to extract useful information from the remotely 
sensed and ancillary data (Atkinson and Tatnall, 1997; 
Foody and Arora, 1997; Qiu and Jensen, 2004). ANN 
model development can also take place using unsuper- 
vised techniques (Filippi et al., 2010). 


Training an Artificial Neural Network 

In the training phase, the analyst selects specific x, y lo- 
cations in the input image with known attributes (e.g., 
agriculture, upland pine) as training sites (Jensen et al., 
2001; Li and Eastman, 2006; Li, 2008). The per-pixel 
spectral information (e.g., spectral reflectance in red 
and near-infrared bands) and ancillary information 
(e.g., elevation, slope, aspect) for each training site is 
then collected and passed to the input layer of the neu- 
ral network. At the same time, the true target (class) 
value of this exact location (e.g., agriculture) is sent to 
the output layer by assigning the neuron representing 
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FIGURE 10-12 Mathematical model of a neuron. The output of the neuron is y, which is a function of the summation of the 
input values (x;) multiplied times their individual weights (w;) in conjunction with an activation function (f). A sigmoid activa- 


tion function is used in this example. 


this class a membership value of 1, while all the other 
output neurons are assigned a value of 0. 


It is important to remember that neural network train- 
ing based on the examples obtained from an image and 
other ancillary data acquired at a specific time and lo- 
cation may be applicable only in the immediate geo- 
graphic area and perhaps only for a given season. 
Thus, they may not be extendable through space or 
time. 


Learning is usually accomplished by adjusting the 
weights using a back-propagation algorithm (Brown et 
al., 2008). For each training example, the output of the 
network is compared with the true target (class) value. 
The difference between the target and output value is 
regarded as an error and is passed back to previous 
layers in the network to update the connection weights. 
The magnitude of the adjustment is proportional to 
the absolute values of the error (discussed shortly in 
the mathematics section). After many iterations, the 
root mean square (RMS) error diminishes to a small 
value less than a predefined tolerance, and further iter- 
ations will not improve the performance of the network 
(Jensen et al., 1999). At this time, the system achieves 
convergence and the training process is completed. The 
rules inherent in the examples are stored in the hidden 
weights in the hidden layer(s) for use in the testing 
(classification) phase. 


Testing (Classification) 

During the fest or classification phase, the spectral and 
ancillary characteristics of every pixel in the scene are 
passed to input neurons of the neural network. The 
neural network evaluates each pixel using the weights 
stored in the hidden layer neurons to produce a pre- 
dicted value for every neuron of the output layers. The 


value obtained for every output neuron is a number be- 
tween 0 and | that gives the fuzzy membership grade of 
the pixel belonging to the class represented by that 
neuron. Defuzzification of these maps using a local 
maximum function leads to a hard classification map 
where each pixel is assigned to a unique class that has 
the highest fuzzy membership grade (Jensen et al., 
2001). 


Mathematics of the Artificial Neural Network 
An artificial neural network (ANN) is defined by neu- 
rons, topological structure, and learning rules. The 
neuron is the fundamental processing unit of an ANN 
for computation. Analogous to the human brain’s bio- 
logical neuron, an artificial neuron is composed of in- 
puts (dendrites), weights (synapses), processing units 
(cell bodies), and outputs (axons) (Hagan et al., 1996). 
Each input x; is multiplied by the scalar weight w; to 
form x,w;, a term that is sent to the “summing unit” of 
the processing unit (Figure 10-12). An offset, b, may be 
added to the total. The summation output: 


P 
net = >, (x;w,) +5, (10.7) 


i=1 


referred to as net input, goes into an activation transfer 
function f that produces scaled neuron output y (be- 
tween 0 and 1, or -1 to 1) through a transform algo- 
rithm. Thus, y can be calculated as (Jensen et al., 
1999): 
P 
y = f(net) =f >, (x;w,) +b]. (10.8) 


b= 


The topological structure of an artificial neural net- 
work defines the overall architecture of the network, 
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FIGURE 10-13 The flow of data (e.g., test, training, imagery), decisions, and iterations associated with a typical back-propa- 
gation multi-layer perceptron (MLP) neural network (based on Hu, X. and Q. Weng, 2009, “Estimating Impervious Surfaces 
from Medium Spatial Resolution Imagery Using the Self-organizing Map and Multi-Layer Perceptron Neural Networks,” 


Remote Sensing of Environment, 113:2089-2102). 


including the framework and interconnection of neu- 
rons that are organized into layers. As previously dis- 
cussed, a typical ANN consists of three or more layers 
(Figure 10-11): one input layer, one output layer, and 
one or more hidden layers (Jensen et al., 2001). Neu- 
rons within and between layers can be connected to 
form a specific user-defined, task-oriented network. 
Decision processes can be duplicated or approximated 
through an ANN’s ability to learn and recall. Learning 
is made possible by feeding the input layer with train- 
ing data. By comparing the current activation of neu- 
rons in the output layer to a desired output response, 
the difference can be obtained and used to adjust 
weights connecting neurons in the network. It is the 
weights that determine the behavior of the network, so 
the goal of learning is to achieve a set of weights that 
will produce an output most resembling the target. 
This adaptive learning process repeats until changes in 
network weights drop below a preset threshold, which 
indicates a user-defined accuracy. Once learned, the 
network can recall stored knowledge to perform either 
classification or prediction on new input data. 


The desirable properties of non-normality and nonlin- 
earity of an ANN can be attributed not only to the net- 
work’s massively parallel distributed structure, but also 
to the transfer function for each neuron (Haykin, 
1994). A neuron is basically a nonlinear device that 
may take continuous and differential functions as the 
transfer function. Therefore, ANNs are capable of 
modeling any complex system, be it physical or hu- 
man. Image classification using ANNs takes advantage 
of this relationship to incorporate non-normally dis- 
tributed numerical and categorical GIS data and image 
spatial information (e.g., texture and context) into the 
process (Qiu and Jensen, 2004). 


There are several commonly used artificial neural net- 
work learning algorithms that can be used to analyze 
remote sensor data, including: 


¢ Feed Forward Multi-layer Perceptron (MLP) Neu- 
ral Network with Back Propagation (BP), 


* Kohonen’s Self-Organizing Map (SOM) Neural 
Network, and 


¢ Fuzzy ARTMAP. 


It is useful to briefly introduce their characteristics. 


Feed Forward Multi-Layer Perceptron (MLP) 
Neural Network with Back Propagation (BP) 
One of the most popular neural networks used in re- 
mote sensing is the feed forward multi-layer perceptron 
(MLP) neural network trained via back-propagation 
(BP) (Filippi and Jensen, 2006; Weng and Hu, 2008; 
Hu and Weng, 2009). MLP neural networks typically 
consist of one input layer, one or more hidden layer, 
and one output layer (Gonzalez Vilas et al., 2011; 
Weng, 2012). Each input neuron represents one band 
of imagery (and/or other type of geospatial data), and 
the number of output nodes corresponds to the num- 
ber of information classes desired. MLP networks us- 
ing BP training algorithms are typically used for 
complex image classification. Classification accuracy is 
very sensitive to the quality and quantity of training 
data selected for each input class (Atkinson and Tat- 
nall, 1997; Khorram et al., 2011). 


A typical MLP neural network with a BP learning al- 
gorithm uses the following algorithm to calculate the 
input that a single node, 7, receives (Weng and Hu, 
2008): 
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FIGURE 10-14 Structure of a multi-layer perceptron (MLP) neural network to predict chlorophyll a concentration of Galician 
Rias in NW Spain using 11 bands of MERIS multispectral data ranging from 442.5 to 865 nm and three geometric characteris- 
tics associated with the images including Sun zenith, view zenith, and the difference between the Sun zenith and view zenith 
(based on Figure 4 from Gonzalez Vilas, L., Spyrakos, E. and J. M. Torres Palenzuela, 2011, “Neural Network Estimation of 
Chlorophyll a from MERIS Full Resolution Data for the Coastal Waters of Galician rias (NW Spain)," Remote Sensing of 


Environment, 115:524—535). 


net, = ei 
i 


where w; denotes the weights between nodes i and j, 
and J; is the output from node i of a sender layer (either 
input or hidden layer). The output from a node / is cal- 
culated as (Hu and Weng, 2009): 


(10.9) 


0; = S(net,). (10.10) 


The function fis usually the nonlinear sigmoidal func- 
tion previously discussed.The flow of information, de- 
cisions, and interactions associated with a typical MLP 
neural network are shown diagrammatically in Figure 
10-13 (Hu and Weng, 2009). Note the adjustment of 
the weights that takes place during back-propagation 
until the error threshold or a user-specified maximum 
number of iterations is reached. The knowledge inher- 
ent in the training data is obtained through the itera- 
tion of the forward and backward passes until all the 
parameters in the network are stabilized (Jensen et al., 
2001). This is signified by the decrease of average 
squared error to a minimum or acceptable level. The 
outputs of an MLP are called activation level maps 
with the number of activation level maps equal to the 


number of output nodes (Hu and Weng, 2009). The 
value of each pixel in the activation level map repre- 
sents the degree to which a pixel belongs to a particular 
class. 


Gonzalez Vilas et al. (2011) used a MLP ANN to map 
chlorophyll a concentration in the coastal waters of 
NW Spain. Their neural network consisted of 1) an in- 
put layer containing 11 MERIS multispectral bands 
and three geometric variables associated with the mul- 
tispectral data (Sun zenith, view zenith, and the differ- 
ence between Sun and view zenith), 2) two hidden 
layers with four nodes each, and 3) a single output neu- 
ron to map chlorophyll a concentration as shown in 
Figure 10-14. 


Jiang et al. (2004) used a back-propagation ANN to 
predict winter wheat yield in northern China. Five in- 
put nodes were used and one hidden layer with eight 
nodes (Figure 10-15). Note that four of the input layer 
nodes are NOAA AVHRR remote sensing—derived 
measurements including the Normalized Difference 
Vegetation Index (NDVI), Absorbed Photosyntheti- 
cally Active Radiation (APAR), canopy surface tem- 
perature (7,), and a Water Stress Index. The fifth input 
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FIGURE 10-15 The characteristics of a back-propagation neural network used to predict winter wheat yield (kg ha‘!) in the 
northern provinces of China using four NOAA AVHRR-derived variables plus the average in situ crop yield per county over the 
last 10 years. Eight hidden layer nodes and a single output layer were used to map the spatial distribution of the winter wheat 
yield (based on Jiang, D., Yang, X., Clinton, N., and N. Wang, 2004, “An Artificial Neural Network Model for Estimating Crop 
Yields using Remotely Sensed Information,” | nernational Joumal of Remote Sensing, 25(9):1723-1732). 


variable (node) is average in situ crop yield over the last 
10 years (by county). A single output layer node was 
used to predict winter wheat yield. 


Kohonen’s Self-Organizing Map (SOM) Neural 
Network 

Kohonen’s self-organizing map (SOM) is another type 
of neural network used in remote sensing research 
(Filippi et al., 2010). A SOM neural network has only 
one input layer and one output layer (Filippi and Jen- 
sen, 2006; Li, 2008). The output layer is known as the 
Kohonen or competitive layer (Li and Eastman, 2006; 
Filippi et al., 2010). The input layer contains neurons 
for each measurement dimension (e.g., image bands or 
other types of geospatial data such as elevation, slope, 
and aspect), and the output layer is usually organized 
as a two-dimension array or lattice of neurons, e.g., 3 x 
3, 4 x 4as shown in Figure 10-16 (Hu and Weng, 2009; 
Salah et al., 2009; Weng, 2012). Each output layer neu- 
ron is connected to all of the neurons in the input layer 
by synaptic weights (Li and Eastman, 2006). Because it 
only uses two layers, the dilemma of determining the 
hidden layer size is avoided. 


Self-organization involves adaptively modifying the 
synaptic weights as a result of input excitations while 


abiding by a learning rule until a useful configuration 
is produced. In the output layer or lattice shown in 
Figure 10-16, output neurons become selectively tuned 
to the presented input patterns during a competitive- 
learning procedure. The topologically-ordered output 
space entails neurons close to one another representing 
similar input patterns. Learning is accomplished by 
first choosing an output neuron that most closely 
matches the presented input pattern, then determining 
a neighborhood of excited neurons around the winner, 
and finally, updating all of the excited neurons (Li, 
2008; Filippi et al., 2010). 


During the labeling stage of an SOM commitment 
(SOM-C) neural network analysis, a competitive layer 
neuron in the n X n output layer is triggered by differ- 
ent patterns. The degree of commitment indicts how 
much an input pattern belongs to a class and is mea- 
sured using the equation (Hu and Weng, 2009): 


Pi) 


> Pi) 


i 
i=1 


(10.11) 
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FIGURE 10-16 Kohonen’s Self-Organizing Map (SOM) neural network has one input layer and one competitive output layer 
consisting of an array (lattice) of neurons (e.g., 4 x 4; 5 x 5). Each output layer neuron is connected to all of the neurons in the 
input layer by synaptic weights (w,). Because it uses only two layers, the dilemma of determining the hidden layer size is avoid- 


ed (based on Salah et al., 2009; Hu and Weng, 2009). 


where P(j) is the proportion of training site of class i, 
triggering neuron /, with Pj) calculated as 


SM) 
Pj) _ a 


1 


(10.12) 


where /,(/) is the frequency of neuron / triggered by pix- 
els labeled as class 7, and N; is the total number of sam- 
ples of class 7 in the training sites. 


Hu and Weng (2009) compared MLP with SOM to es- 
timate impervious surfaces at the sub-pixel level using 
three ASTER images of Marion County, IN, and 
found that the SOM neural network outperformed the 
MLP neural network slightly for each season of image 
data, especially in residential areas. 


Fuzzy ARTMAP Neural Network 


Adaptive Resonance Theory (ART) based neural net- 
works evolved from the biological theory of cognitive 
information processing (Mannan et al., 1998). A gen- 
eralized fuzzy version was introduced by Carpenter et 
al. (1997). The fuzzy ARTMAP architecture designed 
for supervised classification consists of two fuzzy ART 
modules, ART, and ART) as shown in Figure 10-17. 
The ART, module is composed of two layers of pro- 
cessing elements known as F, (input signals) and F, 
(output categories). The F, layer contains the input re- 
mote sensing or other type of geospatial information 
used to train the network. The number of neurons in 
the F, layer is dynamically determined during the pro- 
cess of learning (Mannan et al., 1998) (Figure 10-17). 
A controller uses a minimum learning rule to minimize 
predictive error and maximize predictive generalization 
(Carpenter et al., 1997). This enables the system to 


learn and to determine the number of hidden units 
needed to meet the matching standards (Rogan et al., 
2008; Filippi et al., 2009). 


The ART, module consists of a single Output layer 
that has n neurons associated with the n classes of in- 
terest. There are seven classes in the example shown in 
Figure 10-17. The ART, and ART), modules are con- 
nected by the Map Field layer, which has the same 
number of neurons () as the Output layer. The Map 
Field and Output layers are connected directly. ART- 
MAP has been shown to minimize the three main 
drawbacks of back-propagation networks, including: 
sensitivity to the choice of network parameters, overfit- 
ting, and the need for user intervention during the 
training phase (Mannan et al., 1998). 


Mannan et al. (1998) found that fuzzy ARTMAP was 
more efficient than MLP with the back propagation 
learning in supervised classification of multispectral re- 
motely sensed images. Rogan et al. (2008) compared 
fuzzy ARTMAP with two decision-tree classification 
programs (S-Plus and C4.5) for classification of land 
cover in southern California. They found ARTMAP to 
be robust and accurate for automated, large area 
change monitoring with minimal human intervention 
required in the classification process. Li (2008) put 
forth non-parametric measures associated with fuzzy 
ARTMAP to handle spatial uncertainty in remote 
sensing classification. 


Advantages of Artificial Neural 
Networks 


Characteristics of selected artificial neural networks 
are listed in Table 10-3. Artificial neural networks have 
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FIGURE 10-17 The architecture of a typical fuzzy ARTMAP neural network consisting of two parts, ART, and ARTy. In this ex- 
ample, there are three input sources of geospatial information in the F, Input layer (and their complement). The number of 
neurons in the F layer grows dynamically during the learning process. The number of Map Field layers is the same as the de- 
sired number of Output layers; in this case 7 (based on Figure 1 from Li, Z., 2008, “Fuzzy ARTMAP Based Neurocomputational 
Spatial Uncertainty Measures,” Photogrammetric Engineering & Remote Sensing, 74(12):1573-1584). 


been employed to process multispectral remote sensing 
images and often achieve improved accuracy compared 
to those obtained using traditional statistical methods 
(Filippi and Jensen, 2006; Hu and Weng, 2009; Filippi 
et al., 2010; Khorram et al., 2011). These successes are 
underpinned by many salient characteristics of neural 
networks, including: 


¢« A neural network makes no a priori assumptions 
about normal and linear data distribution due to its 
operation in a nonparametric fashion (Rogan et al., 
2008; Weng and Hu, 2008; Filippi et al., 2010). 
Neural networks are able to learn from existing 
examples adaptively, which makes the classification 
objective. 

The nonlinear patterns are “learned” from the 
empirical examples instead of “prespecified” by an 
analyst based on prior knowledge of the datasets 
(Hagan, et al., 1996). 

The noisy information inevitably included in the 
examples supplies a trained neural network with the 
ability to generalize, which makes neural networks 
robust solutions in the presence of previously unseen, 
incomplete, or imprecise data. The knowledge in a 


traditional expert system that must be extracted from 
knowledgeable experts of a domain area may be 
subjective and incomplete. This is because the experts 
may have a biased or even incorrect understanding of 
reality, they may not be aware of underlying rules 
they have used, and they may have difficulty 
articulating these rules. Conversely, knowledge in a 
neural network is acquired through learning by 
empirical (real-world) training examples. Although 
experts are essential in selecting and preparing 
sample training data, generally their personal biases 
are excluded from the knowledge acquisition process. 
Knowledge in an expert system is represented by 
logical rules made up of binary predicates. Numerical 
attributes have to be converted to binary true/false 
statements, which may cause a large amount of 
information to be lost in the simplification process 
(Gahegan, 2003). On the other hand, a neural 
network can embrace data in all formats as long as 
the data are converted to a numeric representation 
(Brand and Gerritsen, 2004). 

Most rule-based expert systems fail to generalize a 
predictable inference if an appropriate match with the 
perfect rules that must be articulated by experts 
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TABLE 10-3 Characteristics of selected artificial neural networks (based on observations from Ji, 2000; Qiu and Jensen, 2004; 
Li and Eastman, 2006; Li, 2008; Hu and Weng, 2009; Weng and Hu, 2008; Filippi et al., 2010; Weng, 2012). 


Neural Network 


Advantages and Disadvantages 


Multi-Layer Perceptron (MLP) 
with Back-Propagation (BP) 


Often consists of one input layer, one hidden layer, and one output layer. 


MLP is very sensitive to the structure of the network, especially the number of hidden layers and the 
number of nodes in the hidden layer. Determining the optimum number of hidden layer nodes is 
problematic. There is no well-accepted determination method. 


MLP training data must contain both presence and absence data and the output must contain both 
true and false information. 


The training stops at a local minimum instead of a global minimum. 
The back-propagation (BP) learning process is time-consuming. 

BP training process is not consistent and may result in overfitting. 
Slow learning compared to SOM. 


e Black-box working method. 


Classification phase is faster than SOM. 


Map (SOM) hidden layer is avoided. 


e Learns rapidly. 
e Classifies slowly. 


accuracy. 


Kohonen’s Self-Organizing e Consists of one input and one output layer. Therefore, the dilemma of determining the size of the 


e Not necessary to provide both presence and absence information. Presence information is sufficient. 
e  Itis not affected by local minima in the training process. 


e Classification results are more consistent than MLP. 
e Different SOM feature map sizes (e.g., 4 x 4, 5 x 5, etc.) have a significant impact on classification 


e The number of samples for each training class needs to be balanced. 


¢ Minimizes overfitting. 


Fuzzy ARTMAP e Requires little intervention during the training phase. 


cannot be obtained. Conversely, the knowledge in a 
neural network derived from real-world training 
examples inevitably contains noise no matter how 
careful the examples are selected. Therefore, neural 
networks are good at generalizing both discrete and 
continuous data and have a capability to interpolate 
or adapt to the patterns never seen in the training 
process. Thus, neural networks are tolerant of noise 
and missing data and attempt to find the best fit for 
input patterns (Russell and Norvig, 2010). 


¢ Finally, neural networks continuously adjust the 
weights as more training data are provided in a 
changing environment. Thus, they continuously 
learn. 


Limitations of Artificial Neural 
Networks 


Despite the useful performance of neural networks in 
image classification, it is usually difficult to explain in a 
comprehensive fashion the process through which a 
given decision or output has been obtained from a neu- 
ral network (Qiu and Jensen, 2004). There exists an in- 
herent inability to represent knowledge acquired by the 
network in an explicit form with simple “if—then” rules. 
The rules of image classification and interpretation 
learned by the neural network are buried in the weights 
of the neurons of the hidden layers. It is difficult to in- 
terpret these weights due to their complex nature. 


Therefore, a neural network is often accused of being a 
black box (Qiu and Jensen, 2004). 


For these reasons, neural networks may not be accept- 
ed as trusted solutions in some critical real-world ap- 
plications. Using neural networks, an analyst might 
find it difficult to gain an understanding of the prob- 
lem at hand because of the lack of an explanatory ca- 
pability to provide insight into the characteristics of 
the dataset. For the same reason, it is difficult to incor- 
porate human expertise to simplify, accelerate, or im- 
prove the performance of image classification; a neural 
network always has to learn from scratch. For neural 
networks to be widely applied in complex remote sens- 
ing image classification tasks, an explanation capabili- 
ty should eventually be an integral part of the 
functionality of a trained neural network. A possible 
solution is to combine neural networks with fuzzy logic 
so that the knowledge residing in the hidden neurons 
of the network can be extracted in the form of fuzzy 
“if-then” rules (Qiu and Jensen, 2004). 
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INFORMATION EXTRACTION 


USING IMAGING SPECTROSCOPY 


Imaging spectrometry is: 


the simultaneous acquisition of images in many 
relatively narrow, contiguous and/or noncontig- 
uous spectral bands throughout the ultraviolet, 
visible, and infrared portions of the electromag- 
netic spectrum. 


The dense set of spectral measurements is collected us- 
ing an imaging spectrometer, commonly referred to as 
a hyperspectral remote sensing instrument. Many, al- 
though not all, surface materials such as vegetation, 
water, rocks, etc., have diagnostic absorption features 
that are only 10 — 20 nm wide. Therefore, hyperspectral 
sensors that acquire data in many contiguous or non- 
contiguous 10 — 20-nm wide bands throughout the 400 
— 2,500-nm region of the electromagnetic spectrum 
sometimes capture spectral data with sufficient resolu- 
tion for the direct identification of those materials (Im 
and Jensen, 2008). The value of using a hyperspectral 
imaging spectrometer lies in its ability to provide a 
high-resolution reflectance (or emittance) spectrum for 
each pixel in the image (Goetz et al., 1985; Schaepman 
et al., 2009; Chiu et al., 2011). 


ve Overview 


This chapter first reviews the differences between pan- 
chromatic, multispectral, and hyperspectral remote 
sensing instruments. Because of the unique nature of 
the hyperspectral data, special-purpose digital image 
processing algorithms are used to extract useful infor- 
mation. The pre-processing steps required to analyze 


Source of AVIRIS data: NASA 


hyperspectral data are reviewed, including image quali- 
ty assessment, radiometric calibration to scaled surface 
reflectance, geometric correction and dimensionality 
reduction. Special-purpose information extraction 
techniques are reviewed, including: the identification of 
spectrally-pure endmembers, spectral angle mapping, 
subpixel classification (linear spectral unmixing), spec- 
tral library matching techniques, the use of various hy- 
perspectral indices, and derivative spectroscopy. 


Panchromatic, Multispectral 
and Hyperspectral Data 
Collection 


Optical remote sensing systems typically collect pan- 
chromatic, multispectral, and hyperspectral data with- 
in the region from 400 — 2,500 nm. There are also now 
hyperspectral thermal-infrared sensors. 


Panchromatic 


Some remote sensing systems record reflected or emit- 
ted electromagnetic energy in a single broad wave- 
length interval, e.g., from 500 to 700 nm (includes 
green and red wavelength energy) or 500 to 900 nm (in- 
cludes green, red, and near-infrared energy). This is 
typically called a panchromatic band or panchromatic 
remote sensing. Considerable panchromatic (black- 
and-white) aerial photography has been collected 
through the years and used extensively in stereoscopic 
photogrammetric applications. Several satellite remote 
sensing systems have only one panchromatic band 
(e.g., WorldView-1 from 400 — 900 nm at 0.5 x 0.5 m 
spatial resolution; EROS B from 500 — 900 nm at 0.7 x 
0.7 m). Numerous multispectral sensing systems have a 
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single panchromatic band, which is often at much 
higher spatial resolution than the other data collected 
(e.g., Landsat ETM*, Landsat 8, SPOT 5, GeoEye-1). 


Multispectral 


Most satellite and airborne multispectral remote sens- 
ing systems collect data in 3 to 10 spectral bands with 
relatively broad bandwidths (e.g., natural color and 
color-infrared digital frame cameras, Landsat TM, 
ETM”, Landsat 8, SPOT 6, GeoEye-1, WorldView-2, 
QuickBird, Pleiades). 


Hyperspectral 


Hyperspectral remote sensing systems typically collect 
data in >10 spectral bands with relatively narrow 
bandwidths (e.g., Gao et al., 2009; Thenkabail et al., 
2011). However, there is no agreed upon minimum num- 
ber of bands or bandwidth dimension required for a data- 
set to be considered hyperspectral. The collection of 
spectral reflectance and emittance information in hun- 
dreds of spectral bands requires different approaches 
to sensor system design. Two basic approaches to im- 
aging spectrometry are shown in Figure 11-1. The 
“whiskbroom” scanner linear array approach is analo- 
gous to the scanner approach used for Landsat ETM’, 
except that radiant flux from within the IFOV is passed 
onto a spectrometer, where it is dispersed and focused 
onto a linear array consisting of >10 detectors (Figure 
11-la). The terrain within each IFOV (i.e., pixel) is 
sensed in as many spectral bands as there are detector 
elements in the linear array. 


Other imaging spectrometers make use of linear and 
two-dimensional area arrays of detectors. In this situa- 
tion, there is a dedicated column of spectral detector 
elements for each cross-track pixel in the scene (Figure 
11-1b). This “pushbroom” configuration improves im- 
age geometry and radiometry because a scanning mir- 
ror is not used, allowing each detector to dwell longer 
on an IFOV, thereby resulting in a more accurate re- 
cording of the radiant flux as it exits the landscape. 


Satellite Hyperspectral Sensors 


Satellite hyperspectral remote sensing systems include 
(Table 11-1): 


« NASA’s Moderate Resolution Imaging Spectrome- 
ter (MODIS) onboard the Terra and Aqua satellites, 
collects data in 36 co-registered spectral bands: 20 
bands from 0.4— 3 Xm and 16 bands from 3 — 15 Xm. 

« NASA’s EO-1 Hyperion sensor with 198 useful 
bands out of 220 that are calibrated, including chan- 
nels 8 — 57 for the VNIR and channels 77 — 224 for 
the SWIR extending from 400 — 2,500 nm with a 
spectral resolution of 10 nm. 


Hyperspectral Remote 
Sensing based on Linear 
and Area Array 
“Pushbroom” 
Technology 


Multispectral or 
Hyperspectral Remote 
Sensing based 
on Linear Array 
“Whiskbroom” 
Technology 


NIR 


Dispersing 


n bands element 


Detectors 


Rotating mirror NIR 


Blue 
n bands 


FIGURE 11-1 a) A multispectral or hyperspectral “whisk- 
broom” remote sensing system based on linear array detec- 
tor technology and a scanning mirror (e.g., AVIRIS). b) A 
hyperspectral instrument based on linear and area array 
“oushbroom” technology. 


¢ European Space Agency (ESA) Medium Resolution 
Imaging Spectrometer (MERIS) on Envisat used 
“pushbroom” technology to collect 15 selectable 
bands in the 390 — 1,040 nm region with bandwidths 
programmable between 2.5 and 30 nm. MERIS is 
no longer operational (ESA MERIS, 2014). 


« NASA’s Hyperspectral Infrared Imager (HyspIRI) 
is scheduled to include two instruments (NASA 
HyspIRI, 2014): 1) an imaging spectrometer that 
senses in the visible to short-wave infrared (VSWIR: 
380 — 2,500 nm) in 10 nm contiguous bands, and 2) 
a multispectral imager that collects data in 8 bands 
in the thermal-infrared (TIR) region; one band in 
the region from 3 — 5 Xm to measure hot targets and 
seven bands in the region from 7 — 12 Xm (Ramsey 
et al., 2012). The VSWIR and TIR sensors will have 
a spatial resolution of 60 < 60 m. 


Additional information about these satellite sensor sys- 
tems is found in Chapter 2. 


Airborne Optical Hyperspectral Sensors 


Selected suborbital airborne hyperspectral remote sens- 
ing systems include (Table 11-1): 


e NASA’s Airborne Visible/Infrared Imaging Spec- 
trometer (AVIRIS) uses a whiskbroom scanning 
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Imaging Spectrometry-Derived Spectra 


7 


Scaled Surface ReXlectance 


a. Color composite of 3.4 x 3.4m AVIRIS data 
(RGB = bands 50, 30, 20) from Run 08 Scene 05. 


l AP, 
0.5 1.0 15 2.0 
Wavelength, im 


b. Scaled surface reXectance curve for a 
pixel of loblolly pine (Pinus taeda). 


FIGURE 11-2 a) A color composite image of Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) imagery obtained on 
July 26, 1999, at the Savannah River Site near Aiken, SC, at 3.4 3.4 m spatial resolution. Original imagery courtesy of NASA. 
b) Scaled surface reflectance spectrum for a pixel of loblolly pine (Pinus taeda) forest at the location shown in (a). 


mirror and linear arrays of silicon (Si) and indium- 
antimonide (InSb) to record energy in 224 bands, 
each 10 nm wide in the 400 — 2,500 nm region 


* Compact Airborne Imaging Spectrometer 1500 
(CASI-1500) is a pushbroom imaging spectrometer 
that collects data in the region from 380 — 1,050 nm 
and has a 40X total field of view across 1,500 pixels. 
Unlike most other hyperspectral sensors, CASI- 
1500 is spectrally programmable meaning that the 
user can specify which bands are to be collected for 
a specific application (ITRES CASI-1500, 2014). 


¢ SASI-600 collects 100 bands of SWIR hyperspectral 
data from 950 — 2,450 nm at 15-nm intervals with 
600 across-track pixels. SASI-600 data are especially 
useful for geological exploration and plant specia- 
tion (ITRES SASI-600, 2014). 


¢ HyVista HyMap is a whiskbroom hyperspectral 
sensor with 128 bands from 450 — 2,480 nm with 
contiguous spectral coverage (except in the atmo- 
spheric water vapor bands near 1,400 and 1,900 nm) 
with average spectral sampling intervals between 13 
—17nm (HyVista Hymap, 2014). 

« DAIS 7915 (DLR), manufactured by GER, Inc., 
was a 79-band whiskbroom sensor sensitive to the 
VIS-NIR-SWIR-TIR regions from 400 — 1,300 nm 
(Ben-Dor et al., 2002; GAC, 2013). 


¢ AISA Eagle hyperspectral sensor records 244-488 
bands in the region from 380 — 2,500 nm with a 3.3 
nm average spectral sampling interval (Specim 
AISA, 2014). 

¢ AISA FENIX hyperspectral sensor records 620 
bands in the region from 380 — 2,500 nm with a 6 —7 


nm average spectral sampling interval (Specim 
AISA, 2014). 


Airborne Thermal-Infrared Hyperspectral 
Sensors 


ITRES Research, Ltd., markets mid-wavelength infra- 
red (MASI-600) and thermal-infrared (TASI-600) hy- 
perspectral instruments (Table 11-1): 


« MASI-600 is an airborne mid-wavelength thermal- 
infrared hyperspectral sensor that records 64 bands 
in the 4—5 Xm range with a 40M across-track field of 
view encompassing 600 pixels ITRES MASI-600, 
2014). 


¢ TASI-600 is a pushbroom hyperspectral thermal- 
infrared sensor system that collects 32 bands in the 
region from 8 to 11.5 Km with a 40X across-track 
field of view encompassing 600 pixels (ITRES 
TASI-600, 2014). 


Traditional broadband remote sensing systems such as 
Landsat TM and SPOT FR V undersample the infor- 
mation available from a reflectance spectrum by mak- 
ing only a few measurements in spectral bands 
sometimes several hundred nanometers wide. Con- 
versely, imaging spectrometers sample at close intervals 
(bands typically on the order of 10 — 20 nm wide) and 
have a sufficient number of spectral bands to allow 
construction of spectra that closely resemble those ac- 
quired by laboratory spectroradiometers. Analysis of 
imaging spectrometer data allows the extraction of a 
detailed spectrum for each picture element in the im- 
age. Such spectra may allow direct identification of 
specific materials within the IFOV of the sensor based 
upon their reflectance/absorptance characteristics, in- 
cluding asphalt, concrete, rocks and minerals, atmo- 
spheric gases, vegetation, snow and ice, and dissolved 
matter in water bodies (e.g., Nolin and Dozier, 2000; 
Chiu et al., 2011). For example, Figure 11-2 depicts the 
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TABLE 11-1 Characteristics of selected hyperspectral remote sensing systems (Dalponte et al., 2009; DLR ROSIS, 2014; 
HyVista HyMap, 2014; ITRES, 2014; Plaza et al., 2009; Specim AISA, 2014; NASA, and others). 


Number Maximum Spectral Spectral Range 
Sensor Source Platform of Bands Resolution (nm) (Xm) 
EO-1 Hyperion NASA Goddard SFC Satellite 220 10 0.4—2.5 
MODIS NASA Goddard SFC Satellite 36 40 0.4-14.3 
MERIS European Space Agency Satellite 15 2.5=30 0.390-1.04 
CRIS Proba European Space Agency Satellite up to 63 1.25 0.415-1.05 
AVIRIS NASA Jet Propulsion Lab Airborne 224 10 0.4—2.5 
HYDICE Naval Research lab Airborne 210 7.6 0.4—2.5 
PROBE-1 Earth Search Sciences Airborne 128 nz 0.4—2.45 
CASI-1500 ITRES Research Ltd. Airborne 288 235) 0.4-1.05 
SASI-600 ITRES Research Ltd. Airborne 100 15 0.95-2.45 
TASI-600 ITRES Research Ltd. Airborne 32 250 8-11.5 
HyMap HyVista, Inc. Airborne 128 15 0.44—-2.5 
ROSIS DLR Airborne 115 4 0.4-0.9 
EPS-H GER, Inc. Airborne 133 0.67 0.43-12.5 
EPS-A GER, Inc. Airborne 31 2.3 0.43-12.5 
DAIS 7915 GER, Inc. Airborne 79 15) 0.43-12.3 
AISA Eagle Specim, Inc. Airborne 244-488 8,3) 0.4-0.97 
AISA Owl Specim, Inc. Airborne 96 100 7.7-12.3 
AISA Hawk Specim, Inc. Airborne 320 8.5 0.99-2.45 
AISA Dual Specim, Inc. Airborne 500 2S) 0.4—2.45 
AISA FENIX Specim, Inc. Airborne 620 3.5 and 12 0.38-2.5 
MIVIS Daedalus Airborne 102 20 0.43-12.7 
AVNIR OKSI Airborne 60 10 0.43-1.03 


scaled surface reflectance characteristics of hyperspec- 
tral data obtained over a loblolly pine (Pinus taeda) 
forest on the Savannah River Site near Aiken, SC, in 
the wavelength interval from 400 — 2,500 nm. 


Steps to Extract Information 
from Hyperspectral Data 


Scientists have developed digital image processing 
methods that may be applied in a relatively straightfor- 
ward sequence to extract useful information from hy- 
perspectral remote sensor data (e.g., Eismann et al., 


2012; Chang et al., 2013). Many of the operations iden- 
tified in Figure 11-3 are discussed in this chapter. 


The analysis of hyperspectral data usually requires so- 
phisticated digital image processing software (e.g., EN- 
VI, ERDAS Imagine; IDRISI, PCI Geomatica, 
VIPER Tools). This software must be able to: 


1. Calibrate (convert) the raw hyperspectral at-sensor 
radiance to apparent surface reflectance and ideally 
to surface reflectance. This requires the removal of 
atmospheric attenuation, topographic effects due to 
slope and aspect, and any sensor system—induced 
electronic anomalies. 
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General Steps Used to Extract 
Information from Hyperspectral Data 


State the nature of the information extraction problem. 
* Specify the geographic region of interest. 
* DeMne the classes or biophysical materials of interest. 
Acquire appropriate remote sensing and initial ground reference data. 
* Select remotely sensed data based on the following criteria: 
- Remote sensing system considerations: 
- Spatial, spectral, temporal, and radiometric resolution 
- Environmental considerations: 
- Atmospheric, soil moisture, phenological cycle, etc. 
* Obtain initial ground reference data based on: 
- a priori knowledge of the study area 
Process hyperspectral data to extract thematic information. 
* Subset the study area from the hyperspectral data Xight line(s). 
* Conduct initial image quality assessment: 
- Visual individual band examination 
- Visual examination of color composite images 
- Animation 
- Statistical individual band examination; signal-to-noise ratio 
* Radiometric correction: 
- Collect necessary in situ spectroradiometer data (if possible) 
- Collect in situ or environmental data (e.g., using radiosondes) 
- Perform pixel-by-pixel correction (e.g., ACORN, FLAASH) 
- Perform band-by-band spectral polishing 
- Empirical line calibration 
* Geometric correction/rectiXcation: 
- Use onboard navigation and engineering data (GPS 
and Inertial Navigation System information) 
- Nearest-neighbor resampling 
* Reduce the dimensionality of hyperspectral dataset: 
- Minimum Noise Fraction (MNF) transformation 
* End-member determination—locate pixels with relatively pure 
spectral characteristics: 
- Pixel Purity Index (PPI) mapping 
- N-dimensional end-member visualization 
* Methods of mapping and matching using hyperspectral data: 
- Spectral Angle Mapper (SAM) 
- Subpixel classiXcation (spectral mixture analysis) 
- Spectroscopic library matching techniques 
- Matched Klter or mixture-tuned matched Klter 
- Machine-learning (e.g.,Support Vector Machine, Regression Tree) 
- Indices developed for use with hyperspectral data 
- Derivative spectroscopy 
Perform accuracy assessment (Chapter 13). 
* Select method: 
- Qualitative conXdence-building 
- Statistical measurement 
* Determine number of observations required by class. 
* Select sampling scheme. 
* Obtain ground reference test information. 
* Create and analyze error matrix: 
- Univariate and multivariate statistical analysis 
Accept or reject previously stated hypothesis. 
Distribute results if accuracy is acceptable. 


FIGURE 11-3 A generalized flow chart useful for the extraction of quantitative biophysical and thematic information from hy- 
perspectral remote sensor data. 
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AVIRIS Low-Altitude Imaging Spectroscopy 
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FIGURE 11-4 Conceptual diagram of low-altitude imaging spectroscopy performed using the NASA Jet Propulsion Labora- 
tory's Airborne Visible/Infrared Imaging Spectrometer (AVIRIS). The scanner mirror focuses radiant flux onto linear arrays that 
contain 224 detector elements with a spectral sensitivity ranging from 400-2,500 nm. A spectrum of radiance (L) is obtained 
for each picture element. The radiance data can be processed to yield percent reflectance information. Most AVIRIS over- 
flights obtain data at 20 20 m pixels. Sometimes the AVIRIS sensor is flown on other aircraft and at different altitudes above 
ground level. This particular AVIRIS scene of the Savannah River Site near Aiken, SC, was acquired at 11,155 ft. above sea lev- 
el in a De Havilland DHC-6 Twin Otter on July 26, 1999. This resulted in 3.4 X 3.4 m spatial resolution data. The at-sensor ra- 
diance (L) for a pixel of Bahia grass is shown. The image displayed is a color composite of bands 50 (near-infrared), 30 (red), 


and 20 (green). AVIRIS original imagery courtesy of NASA. 


2. Analyze the calibrated remote sensing—derived 
scaled surface reflectance data to determine its 
constituent materials by a) deriving endmembers 
from the hyperspectral data itself, b) comparing in 
situ spectroradiometric data obtained in the field at 
the time of the overflight with hyperspectral image 
characteristics, and/or c) comparing hyperspectral 
image characteristics with a library of laboratory- 
based spectra such as that provided by Johns 
Hopkins University, the U.S. Geological Survey, 
and NASA’s Jet Propulsion Laboratory (Baldridge 
et al., 2009; ASTER Spectral Library, 2014). 


To understand how to analyze hyperspectral data, it is 
useful to examine the image processing methods in the 
context of real-world applications. The chapter first de- 
scribes how AVIRIS remote sensor data were analyzed 
to detect possible stress associated with Bahia grass— 
covered clay caps on the U.S. Department of Energy 
Savannah River Site (SRS) near Aiken, SC. AVIRIS 
sensor system characteristics are summarized in Table 
11-1 and Figure 11-4. In 1999, the AVIRIS sensor was 
also flown in low-altitude mode on a NOAA De Havil- 
land DHC-6 Twin Otter aircraft that acquired data at 
an altitude of 11,155 ft AGL. This resulted in 224 
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bands of hyperspectral data with a nominal spatial res- 
olution of approximately 3.4 <x 3.4 m. Figure 11-4 de- 
picts the at-sensor radiance (L) associated with a single 
pixel of Bahia grass. 


High spatial resolution AVIRIS data of grassland in 
Monticello, UT, is then analyzed to classify land cover 
and predict continuous variables (e.g., LAI). Finally, 
AISA Eagle hyperspectral data of forest treatment 
plots on the SRS are analyzed to measure biophysical 
variables (LAI, biomass) and vegetation chemical 
characteristics (e.g., N, Ca). 


Select Study Area from Flight 
Lines 


Both public (e.g., NASA Jet Propulsion Laboratory) 
and commercial (e.g., HyMap, CASI-1500) data pro- 
viders usually pre-process the hyperspectral data be- 
fore transferring it to the user for image analysis 
(Prost, 2013). This preliminary preprocessing is per- 
formed so that the user can look at (browse) the con- 
tent of the imagery collected and then select a subset 
for detailed analysis. For example, personnel at NA- 
SA’s Jet Propulsion Laboratory Data Facility prepro- 
cess the raw digital AVIRIS data to remove basic 
radiometric and geometric errors. The preliminary 
geometric corrections are applied to each flight line 
(often referred to as a run) of AVIRIS data. Each 
AVIRIS flight line is then subset into individual scenes. 
The individual scenes may be viewed via the Internet as 
quick-look image files. For example, scenes 04, 05, and 
06 extracted from run 08 collected over the Savannah 
River Site on July 26, 1999, are shown in Figure 11-5. 
There is no overlap of the successive scenes. Therefore, 
it is relatively straightforward to mosaic the scenes into 
a single flight line. A datacube of run 08 scene 05 is 
shown in Figure 11-5d. The dark regions of the 224- 
band datacube correspond to atmospheric absorption 
bands present especially at 1,400 and 1,900 nm. 


| ritial Image Quality 

Assessment 
It is important to perform an initial assessment of the 
quality of the hyperspectral data. A number of meth- 
ods can be used to assess image quality including visu- 
al examination of multiple-band color composites, 
inspection of individual bands of imagery, viewing an 
animated movie of the individual bands, and statistical 
evaluation of individual bands including a quantitative 


evaluation of individual band signal-to-noise ratios 
(SNRs). 
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Visual Examination of Hyperspectral 
Color Composite Images 


One of the most useful ways to determine if hyperspec- 
tral imagery has information of value is to select repre- 
sentative bands from the near-infrared (e.g., AVIRIS 
band 50), red (e.g., band 30), and green (e.g., band 20) 
portions of the spectrum and create a false-color com- 
posite of the scene as shown in Figure 11-5. Hopefully, 
the individual bands are co-registered and contain 
spectral information of value. The analyst can select 
any color composite of three bands to evaluate. Look- 
ing at numerous three-band color composites from the 
224 possible bands can be a tedious process. However, 
the exercise does provide valuable qualitative informa- 
tion about the individual scenes and bands in the hy- 
perspectral dataset. Sometimes this is the only way to 
determine if a band is geometrically misregistered (off- 
set) when compared to the other bands. 


Visual Individual Band Examination 


Although it is a tedious process, sometimes there is 
simply no substitute for painstakingly viewing each 
band of data to judge its quality. Many bands appear 
crisp and sharp with good grayscale tonal variation. 
For example, Figure 11-6a depicts AVIRIS band 50 
(centered at 0.8164 Xm) from run 08 scene 05. Com- 
pare this image with images of AVIRIS band 112 
(1.405 Xm) and band 163 (1.90 Xm) shown in Figure 
11-6bc. These images were obtained in two of the 
prominent atmospheric absorption windows. There- 
fore, it is not surprising that they are of poor quality. 
The analyst normally keeps a list of bands that exhibit 
considerable atmospheric noise (i.e., a bad bands list). 
Such bands are often deleted from further analysis, as 
described in subsequent sections. However, it is impor- 
tant to remember that the information in the atmo- 
spheric absorption bands may be used by certain 
atmospheric correction algorithms. 


It is very important to determine early in a project if 
certain bands or particular regions within a given band 
contain null data values (e.g., values of —9,999) or if 
there are serious line dropouts (i.e., an entire line has a 
value of -9,999). Also, an examination of the individu- 
al bands and color composites provides information on 
whether there are serious geometric problems with the 
data such as extreme image warping. Deep notches 
along the edges of a scene often indicate spatial image 
distortion, which must be corrected during the geomet- 
ric rectification phase of the project. 


Animation 


Hyperspectral datasets usually contain hundreds of 
bands. Therefore, most digital image processing sys- 
tems designed for hyperspectral analysis have an image 
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Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Data 
of the Savannah River Site Obtained July 26, 1999 


Mixed Waste Management Facility 


H-area Seepage Basin 


a. Run 08 scene 04. b. Run 08 scene 05. c. Run 08 scene 06. 


Image on top of 
the datacube is a 
color composite of 
three of the 224 bands 
(RGB = bands 50, 30, 20). 
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FIGURE 11-5 a-c) Color- infrared color- compos ites of three AVIRIS scenes (04, 05, and 06) from run 08. The color composites 
represent a display of just three of the 224 AVIRIS bands (RGB = bands 50, 30, 20). The AVIRIS sensor was flown onboard a De 
Havilland DHC-6 Twin Otter aircraft at 11,150 ft. (3.4 km) above sea level. This resulted in pixels that were 3.4 3.4 m. Each 
scene is 512 X 746 pixels in dimension. The data were preprocessed at JPL to remove fundamental geometric and radiomet- 
ric errors. Pixels in the black border regions contain null values (-9,999). AVIRIS imagery courtesy of NASA. d) Hyperspectral 
datacube of run 08 scene 05. All 224 bands are displayed. Dark areas in the datacube represent atmospheric absorption bands. 
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Assessing the Quality of Individual AVIRIS Scenes Band by Band 


a. Run 08 scene 05 
band 50 (0.8164 Xm). 


b. Run 08 scene 05 
band 112 (1.405 Xm). 


c. Run 08 scene 05 
band 163 (1.90 Km). 


FIGURE 11-6 Selected individual bands of AVIRIS data from run 08 scene 05. a) An image of band 50 after contrast stretch- 
ing. b) A contrast-stretched image of band 112. c) A contrast-stretched image of band 163. The near-infrared band 50 image 
is relatively free of attenuation, while bands 112 and 163 exhibit the deleterious effects of atmospheric attenuation. 


animation function whereby the analyst selects a cer- 
tain time rate at which individual bands are displayed 
on the screen, e.g., every 5 seconds. Examination of hy- 
perspectral bands in this manner allows the analyst to 
1) identify individual bands that have serious atmo- 
spheric attenuation or electronic noise problems, as 
previously discussed, and 2) determine if any misregis- 
tration (offset) of bands exists. 


Occasionally, linear features in the scene such as roads 
or the edges of water bodies may appear to shift slight- 
ly when you are viewing the animated bands. This does 
not necessarily indicate that the bands are misregis- 
tered, but rather that the detectors in one band are 
more or less sensitive than those in an adjacent band. 
Sometimes with AVIRIS data there is a slew effect in 
the detector array readout, which translates to a shift 
of the IFOV in areas of high spatial frequency. If the 
displacement appears to be relatively severe, then a 
more detailed examination should be conducted. This 
might entail creating color composites of multiple 
bands to determine if there is actually band-to-band 
misregistration. 


Statistical Individual Band 
Examination 


Examination of the univariate statistics (e.g., mean, 
median, mode, standard deviation, range) of the indi- 
vidual bands can be helpful when assessing image 
quality. For example, if a band’s 16-bit brightness val- 
ues are constrained to a very small range with a very 
small standard deviation, this may indicate a serious 


problem. When a band is believed to have problems, 
there is no substitute for analyzing its univariate statis- 
tics and histogram. In order to detect many absorption 
features associated with materials found in nature, a 
noise level that is approximately an order of magnitude 
smaller than the absorption depth is required. There- 
fore, evaluation of each band’s signal-to-noise ratio is 
of value. 


He Radiometric Calibration 


As discussed in Chapter 9, many remote sensing inves- 
tigations do not concern themselves with radiometric 
correction of the data to remove the deleterious effects 
of atmospheric absorption and scattering. However, to 
use hyperspectral remote sensor data properly, it is 
generally accepted that the data must be radiometrical- 
ly corrected. In addition to removing atmospheric ef- 
fects, this process normally involves transforming the 
hyperspectral data from at-sensor radiance, L, (AW 
cm™~ nm” sr-) to scaled surface reflectance (Mustard 
et al., 2001). This allows the remote sensor—derived 
spectral reflectance data (often referred to as spectra) 
to be quantitatively compared with in situ spectral re- 
flectance data obtained on the ground using a hand- 
held spectroradiometer (Figure 11-7a) or with 
laboratory-derived spectra. Such spectroradiometric 
data are often stored in spectral libraries (e.g., Bal- 
dridge, et al., 2009). 
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In Situ Spectroradiometer Measurements used to Calibrate Hyperspectral Data 


a. Spectroradiometer 
being calibrated. 


b. Calibration panels. 
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FIGURE 11-7 In situ spectroradiometer measurements can be valuable when used to per form absolute atmospheric correc- 
tion. They are especially useful for empirical line calibration and as boost spectra. a) A spectroradiometer being calibrated us- 
ing a Spectralon plate prior to a Bahia grass measurement. b) Calibration panels placed in the study area. c) The location of 
additional in situ spectroradiometer measurements obtained on top of the Low Activity Waste (LAW) building on the Savan- 
nah River Site and adjacent to the building (photographs courtesy of Jensen and Hadley, 2003). 


In Situ Data Collection 


Whenever possible, it is desirable to obtain handheld in 
situ spectroradiometer measurements on the ground 
features of interest at approximately the same time as 
the remote sensing overflight (McCoy, 2005). These 
measurements should be made with well-respected 
spectroradiometers (e.g., Spectron Engineering, Ana- 
lytical Spectral Devices) that cover the same spectral 
range as the hyperspectral remote sensing system (e.g., 
400 — 2,500 nm). Ideally, several calibrated spectroradi- 
ometers are used to collect data of the most important 
materials in the study area at approximately the same 
time of day and under the same atmospheric condi- 
tions as the remote sensing data collection. If this is 
not possible, then data should be collected at the same 
time of day on days before and/or after the remote 
sensing data collection. Of course, each spectroradiom- 


eter measurement taken in the field should be calibrat- 
ed using a standard reference panel (e.g., Spectralon; 
Figure 11-7a). Spectroradiometer measurements made 
in the lab under controlled illumination conditions are 
valuable as long as the materials (especially vegetation 
samples) have been properly maintained and analyzed 
as soon as possible. Jn situ spectroradiometer spectral 
reflectance measurements can be useful in the success- 
ful transformation of hyperspectral data from radiance 
to scaled surface reflectance. If in situ spectroradiome- 
ter measurements cannot be obtained at the time of the 
remote sensing data collection, then it may be possible 
to locate some in situ spectra of the features of interest 
in established digital libraries (e.g., Baldridge, et al., 
2009; ASTER Spectral Library, 2014). 


If possible, a radiosonde launched at the time of re- 
mote sensing data collection can provide very valuable 
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information about atmospheric temperature, pressure, 
relative humidity, wind speed, ozone, and wind direc- 
tion aloft. 


Absolute Atmospheric Correction 


Chapter 6 provides information about both relative 
and absolute atmospheric correction. This section only 
describes aspects of absolute atmospheric correction 
for analysis of hyperspectral data. 


Radiative Transfer-Based Absolute Atmospheric 
Correction 


Ideally, the analyst knows the exact nature of the atmo- 
spheric characteristics above each picture element at 
the time of hyperspectral data collection, e.g., baromet- 
ric pressure, water vapor, amount of atmospheric mo- 
lecular (Rayleigh) scattering, etc. Unfortunately, the 
atmosphere is variable even over relatively short geo- 
graphic distances and time periods. One method of at- 
mospheric correction uses the remote sensing—derived 
radiance data in very selective narrow bands to infer 
information about the atmospheric conditions above 
each pixel. This information is then used to remove at- 
mospheric effects from each pixel. The output of radia- 
tive transfer-based atmospheric correction algorithms 
is scaled surface reflectance data stored in a hyperspec- 
tral image cube that has the same dimensions as the in- 
put image cube (Figure 11-5d). Several robust 
algorithms used to remove the effects of atmospheric 
attenuation from individual pixels of hyperspectral 
data include: 


ATmosphere REMoval (ATREM) (Boardman, 
1998; Gao et al., 1999, 2009; Hatala et al., 2010; 
Center for the Study of Earth From Space at the 
University of Colorado, Boulder), 


Atmospheric CORrection Now (ACORN) (San and 
Suzen, 2010; ImSpec ACORN, 2014), 


Fast Line-of-sight Atmospheric Analysis of Spectral 
Hypercubes (FLAASH) (Agrawal and Sarup, 2011; 
Exelis FLAASH, 2014), 


Quick Atmospheric Correction (QUAC) (Bernstein 
et al., 2008; Agrawal and Sarup, 2011; Exelis 
QUAG, 2014), and 


ATmospheric CORrection (ATCOR) (San and 
Suzen, 2010; Richter and Schlaper, 2012; ERDAS 
ATCOR, 2014). 


Detailed characteristics associated with ACORN, 
FLAASH, QUAC, and ATCOR were provided in 
Chapter 6. 


Alexander Goetz first perceived the need for develop- 
ing an atmospheric correction technique using radia- 
tive transfer modeling for hyperspectral image analysis, 
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given the limitations of the empirical approaches for 
surface reflectance retrievals (Gao et al., 2009). The 
thought was soon realized with the development of the 
Atmosphere Removal algorithm (ATREM) (Gao et 
al., 1993) for retrieving land surface reflectance spectra 
from hyperspectral imaging data using a theoretical 
modeling technique, which simulates explicitly the ab- 
sorption and scattering effects of atmospheric gases 
and aerosols (Gao et al., 2009). The development of 
ATREM was a significant event in the history of imag- 
ing spectrometry. 


ATREM can retrieve surface reflectance from imaging 
spectrometer data without the use of in situ spectrora- 
diometer measurements. It atmospherically corrects 
the data based on water vapor absorption bands at 
0.94 Kim and 1.14 Xm. It also determines the transmis- 
sion spectra of various atmospheric gases and the scat- 
tering effects due to atmospheric molecules and 
aerosols. The measured radiances are divided by solar 
irradiance above the atmosphere to yield apparent sur- 
face reflectance. ATREM continues to be upgraded 
(e.g., it now uses 6S instead of 5S for modeling atmo- 
spheric scattering). ATREM is not a commercial prod- 
uct but has been provided to selected users that have 
study areas with substantial atmospheric gaseous ab- 
sorption features through the Center for the Study of 
Earth from Space, University of Colorado at Boulder 
(Gao et al., 2009). 


To appreciate the importance of atmospherically cor- 
recting hyperspectral data prior to data analysis, con- 
sider the application of an ATREM absolute radiative 
transfer-based correction to run 08 scene 05 of the Sa- 
vannah River Site. An example of a raw AVIRIS lob- 
lolly pine (Pinus taeda) spectrum for a pixel located at 
3,683,437 Northing and 437,907 Easting UTM (sam- 
ple 198, line 123) is shown in Figure 11-8a. The y-axis 
measures at-sensor radiance, L (KW em? nm! sr), 
For all practical purposes, the raw radiance spectra is 
not very informative for Earth surface inquiries. Notice 
the dramatic increase in radiance in the shorter wave- 
length (blue) portion of the spectrum caused by in- 
creased atmospheric Rayleigh scattering. Basically, the 
radiance spectra do not resemble the spectra that 
should be associated with a healthy, photosynthesizing 
pine canopy. 


Application of the atmospheric correction program to 
the raw AVIRIS data resulted in the scaled surface re- 
flectance spectrum shown in Figure 11-8b. The scaled 
reflectance now appears as it should, with chlorophyll 
absorption bands in the blue and red, a peak in the 
green portion of the spectrum, increased reflectance 
throughout the near-infrared, and depressions sur- 
rounding the atmospheric absorption bands. Note that 
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Pixel-by-Pixel Atmospheric Correction of AVIRIS Data 


Loblolly pine 
(Pinus taeda) 


At-sensor Radiance, L (scaled) 


0.5 1.0 1.50 2.0 
Wavelength, 4m 


a. Run 08 scene 05, 
original AVIRIS data. 


2.) 0.5 1.0 


Loblolly pine 
(Pinus taeda) 


1.50 
Wavelength, Xm 


b. Run 08 scene 05, 
atmospherically corrected using ATREM. 


Band-by-Band Atmospheric Correction of AVIRIS Data 
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FIGURE 11-8 a) An example of an original AVIRIS vegetation spectral profile for a single pixel of loblolly pine located at 
3,683,437 Northing and 437,907 Easting in the Universal Transverse Mercator (UTM) projection. The y-axis is in units of radi- 
ance (L). b) The results of applying an absolute atmospheric correction algorithm (ATREM) to the original AVIRIS data. This is 
the scaled surface reflectance data for the loblolly pine pixel. c) The results of applying the Empirical Flat Field Optimal Re- 
flectance Transformation (EFFORT) to the atmospherically-corrected loblolly pine data in (b). This resulted in a mild band- 
by-band, entire-scene correction. Some of the artifacts of the atmospheric correction have been partially removed in the 
EFFORT-corrected spectra. The scaled surface reflectance for a pixel of Bahia grass (Paspalum notatum) in the same scene 


is also plotted. 


discontinuities in the spectra represent atmospheric ab- 
sorption bands that are not plotted. 


The quality of the atmospheric correction and radiomet- 
ric calibration to scaled surface reflectance is very im- 
portant. If the atmospheric correction and radiometric 
calibration are poor, it will be difficult for the analyst 
to compare remote sensing—derived spectra with in 
situ—derived spectra stored in spectral libraries. 


Band-by-Band Spectral Polishing Notice in Fig- 
ure 11-8b that there is noise in the loblolly pine (Pinus 


taeda) spectrum even after the data were atmospheri- 
cally corrected on a pixel-by-pixel basis. This is be- 
cause there is still cumulative error in the hyperspectral 
dataset due to sensor system anomalies and the limited 
accuracy of the standards, measurements, and models 
used and calibrations performed along the signal pro- 
cessing chain. This cumulative error can be as much as 
a few percent in each band of data. 


There are additional techniques that can be used to re- 
move some of this cumulative error as discussed in 
Chapter 6. Normally this involves deriving a relatively 
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mild linear correction that is applied to the entire band 
and the entire scene at one time (i.e., it is not a pixel- 
by-pixel correction). The methods “polish” out some 
of this error in an attempt to improve the accuracy of 
the scaled surface reflectance data (Boardman, 1997). 


ENVI, ACORN, and ATCOR all have spectral polish- 
ing (artifact suppression) capability. The spectral pol- 
ishing modules require atmospherically-corrected data 
as input. In addition, the polishing programs can also 
incorporate in situ spectroradiometer spectra for those 
materials most commonly found in the scene. 


For example, ENVI’s Empirical Flat Field Optimal Re- 
flectance Transformation (EFFORT) program per- 
forms spectral polishing (Hatala et al., 2010; Exelis 
ENVI, 2014; Exelis FLAASH, 2014). EFFORT was 
run using the atmospherically-corrected data previous- 
ly discussed. In addition, two in situ spectroradiometer 
sweet gum (Liguidambar styraciflua) spectra (one rep- 
resenting an average of multiple individual spectra, and 
one unaveraged spectrum) and two loblolly pine spec- 
tra were also input. This set of four (4) spectra was ap- 
plied to the atmospherically-corrected bands. An 
EFFORT treatment was also applied using a laborato- 
ry-derived Bahia grass spectrum, but it was applied to 
only a limited number of visible and near-infrared 
channels due to its limited spectral range (400 — 800 
nm). Bahia grass was judged to be important because 
it is grown on many of the clay-caps. Loblolly pine and 
sweet gum are important because they are the vegeta- 
tion types often impacted downslope of clay-capped 
SRS waste sites. 


These in situ spectral reflectance measurements are 
sometimes referred to as “reality boost spectra.” Fig- 
ure 11-8c depicts the same loblolly pine pixel in the hy- 
perspectral dataset after it was spectrally polished 
using EFFORT. Notice that some artifacts in the at- 
mospheric correction are removed and that the vegeta- 
tion spectrum is improved. The spectral reflectance 
curve for a nearby pixel of Bahia grass (Paspalum nota- 
tum) is also shown in Figure 11-8c for comparison. 
Note the dramatic differences in the reflectance char- 
acteristics between the loblolly pine and Bahia grass, 
especially in the visible and near-infrared portion of 
the spectrum. 


Before running programs like EFFORT, it is important 
that the user avoid wavelength ranges that contain 
noise such as the 1.4 Xm and 1.9 Km water vapor ab- 
sorption bands. Also, the user should experiment with 
the order of the polynomial that is fit to the data. The 
lower the order of the polynomial, the more error sup- 
pression, but the greater the potential loss of valuable 
spectral information. A higher-order polynomial will 
produce a spectrum that fits the data better but also 
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may fit some error features. The example was based on 
a 10th-order polynomial. Programs like EFFORT per- 
form a very mild adjustment to the atmospherically- 
corrected data. An additional method of improving the 
EFFORT spectral polishing is to carefully subset out 
only those areas of greatest importance and use “reali- 
ty boost” spectra specific to that area of interest. 
Adler-Golden et al. (2002) provide an algorithm for re- 
moving shadows in spectral imagery. Pignatti et al. 
(2009) and Hatala et al. (2010) used ATREM and EF- 
FORT polishing to prepare hyperspectral imagery for 
extraction of National Park ecosystem information in 
Italy and Wyoming. 


Absolute Atmospheric Correction using 
Empirical Line Calibration 


As discussed in Chapter 6, it is also possible to use in 
situ spectroradiometric data to atmospherically correct 
hyperspectral data using empirical line calibration. 
This method applies a single adjustment to all the pix- 
els in a single band based on a model that is derived 
from the regression of in situ spectroradiometer mea- 
surements at specific locations (e.g., deep nonturbid 
water bodies, asphalt parking lots, bare soil, concrete) 
with radiance measurements extracted from the hyper- 
spectral data at the same geographic locations. The in 
situ spectral measurements can also come from estab- 
lished spectra libraries such as the comprehensive AS- 
TER spectral library (Baldridge et al., 2009; ASTER 
Spectral Library, 2014). An empirical line calibration 
example was provided in Chapter 6. 


Geometric Correction of 
Hyperspectral Remote 
Sensor Data 


It is important to geometrically correct the remote sen- 
sor data to a known datum and map projection at 
some point in the hyperspectral data analysis chain. 
Some image analysts prefer to perform all the informa- 
tion extraction on unrectified, atmospherically correct- 
ed hyperspectral data, and then at the very end 
geometrically correct the derived information for dis- 
tribution to the public. Other scientists geometrically 
correct the hyperspectral data early in the process be- 
cause they want to relate in situ spectroradiometer 
spectral reflectance measurements at known x, y loca- 
tions in the field with the same locations in the hyper- 
spectral data. It is also useful to geometrically rectify 
the hyperspectral data early in the processing chain if 
field collected data are to be used to train a classifier. 
For example, the GPS x, y coordinates of a lake poly- 
gon inventoried on the ground might be used to extract 
training statistics or identify candidate water end- 
member pixels in a rectified hyperspectral dataset. This 
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operation cannot be performed if the hyperspectral 
data are unrectified. 


The geometric rectification techniques described in 
Chapter 7 may be applied to hyperspectral data. The 
process just takes relatively longer because there are 
more bands to process. Most scientists use nearest- 
neighbor resampling when geometrically correcting hy- 
perspectral data. However, n-term polynomial and 
rubber-sheet image warping techniques sometimes do 
not yield satisfactory results when applied to aircraft- 
derived imaging spectrometer data. In such cases, on- 
board engineering and navigation data are also used. 
Properly configured aerial platforms are usually 
equipped with onboard GPS and inertial navigation 
system (INS) technology, which provide data that can 
be applied to the geometric correction problem. Jensen 
et al. (2011) describe the use of point and Jinear fea- 
tures in hyperspectral imagery that can be used to rec- 
tify the remote sensor data. 


Reducing the Dimensionality 
of Hyperspectral Data 


The number of spectral bands associated with a remote 
sensing system is referred to as its data dimensionality. 
Many remote sensing systems have relatively low data 
dimensionality because they collect information in rel- 
atively few regions of the electromagnetic spectrum. 
For example, the SPOT 1-3 HRV sensors recorded 
spectral data in three coarse bands. The Landsat 7 
ETM” records data in seven bands. Hyperspectral re- 
mote sensing systems such as AVIRIS and MODIS ob- 
tain data in 224 and 36 bands, respectively. 
Ultraspectral remote sensing systems collect data in 
many hundreds of bands or in bands with very small 
bandwidths (Meigs et al., 2008). Ultraspectral sensing 
may eventually enable the identification of very fine 
molecular absorption or emission line features such as 
hydrocarbon gases (e.g., benzene). 


As one might expect, data dimensionality is one of the 
most important issues when analyzing remote sensor 
data. The greater the number of bands in a dataset (1.e., 
its dimensionality), the more pixels that must be stored 
and processed by the digital image processing system. 
Storage and processing consume valuable resources. 
Therefore, significant attention has been given to de- 
veloping methods to reduce the dimensionality of hy- 
perspectral data while retaining the information 
content inherent in the imagery. 


Statistical measures such as the optimum index factor 
(OIF), transformed divergence, and principal compo- 
nents analysis have been used for decades to reduce the 
dimensionality of multispectral data. These methods 


are described in Chapters 7, 8, and 9. Unfortunately, 
these methods generally are not sufficient for reducing 
the dimensionality of hyperspectral data. 


Hyperspectral data contain a tremendous amount of 
redundant spectral information. This is not surprising 
when one considers that the individual channels often 
have a nominal bandwidth of only 10 nm. Thus, in the 
spectral region from 820 to 920 nm we can expect to 
find 10 bands measuring the amount of near-infrared 
radiant flux exiting the Earth’s surface. While there are 
certainly subtle differences in the amount of radiant 
flux recorded in each of these bands, there will proba- 
bly be a significant amount of redundant spectral in- 
formation. Statistical analysis usually reveals that 
many of these 10 bands are highly correlated. There- 
fore, one can use statistical methods to a) delete some 
of the unwanted redundant bands, or b) transform the 
data so that the information content is preserved while 
reducing the dimensionality of the dataset. In addition, 
it is hoped that the selected data dimensionality reduc- 
tion method will also remove some of the noise in the 
hyperspectral dataset. 


Minimum Noise Fraction (MNF) 
Transformation 


A useful algorithm for reducing the dimensionality of 
hyperspectral data and minimizing the noise in the im- 
agery is the minimum noise fraction (MNF) transfor- 
mation (e.g., Belluco et al., 2006; Small, 2012). The 
MNF is used to determine the true or inherent dimen- 
sionality of the hyperspectral data, to identify and seg- 
regate noise in the data, and to reduce the computation 
requirements of further hyperspectral processing by 
collapsing the useful information into a much smaller 
set of MNF images (Boardman and Kruse, 1994). The 
MNF transformation applies two cascaded principal 
components analyses (Chen et al., 2003). The first 
transformation decorrelates and rescales noise in the 
data. This results in transformed data in which the 
noise has unit variance and no band-to-band correla- 
tion. A second principal components analysis results in 
the creation of a) coherent MNF eigenimages that con- 
tain useful information, and b) noise-dominated MNF 
eigenimages. Basically, the noise in the hyperspectral 
dataset is separated from the useful information. This 
is very important because subsequent hyperspectral 
data analysis procedures function best when the hyper- 
spectral data contain coherent, useful information and 
very little noise. 


Both the eigenvalues and the output MNF eigenimag- 
es are used to determine the true dimensionality of the 
data. Application of the MNF transformation often 
results in a significant decrease in the number of bands 
that must be processed during subsequent endmember 
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Hyperspectral Data Dimensionality and Noise Reduction 
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k. Plot of eigenvalues associated with 19 minimum noise 
fraction (MNF) bands and the cumulative amount of 
variance accounted for by the rst 10 eigenimages. 


FIGURE 11-9 a-j) The first 10 eigenimages (MNF bands) extracted from run 08 scene 05. k) A plot of the eigenvalues associ- 
ated with the first 19 MNF eigenimages. Note that as the MNF eigenimage number increments, the eigenvalue and the 
coherent information in the eigenimages decr ease. However, there is still a degr ee of spatial coherency in MNF band 10. 


Original AVIRIS imagery courtesy of NASA. 


analysis. In the case of AVIRIS data, the process can 
result in a reduction from 224 bands to <20 useful 
MNF bands. This number of bands is actually quite 
manageable and exhibits the dimensionality of typical 
multispectral datasets. Therefore, some scientists have 
input the MNF bands that describe most of the vari- 
ance directly into traditional classification algorithms 
(e.g., maximum likelihood) (Underwood et al., 2003). 
In the SRS example, the bands dominated by atmo- 
spheric noise were discarded prior to running the 
MNF transformation. 


MNF output bands that contain useful image informa- 
tion typically have an eigenvalue an order of magni- 
tude greater than those that contain mostly noise. An 
analyst can determine the information content of indi- 
vidual MNF eigenimages by a) displaying them on the 
computer screen and visually analyzing them, or b) 
plotting their eigenvalues or the amount of variance 
explained by each eigenvalue. For example, an MNF 
transformation was applied to AVIRIS run 08 scene 
05. Figure 11-9 depicts the first 10 MNF eigenimages. 
The first 19 MNF bands could be selected for further 
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Masked AVIRIS Hyperspectral Data of the Clay-capped Mixed Waste Management Facility 


a. False-color composite (RGB = bands 50, 30, 20). 


b. MNF false-color composite (RGB = MNF 3, 2, 1). 


FIGURE 11-10 a-b) Only the clay-capped hazardous waste sites associated with the Mixed Waste Management Facility and 
some internal roads are included in the July 26, 1999, AVIRIS imagery. All other land cover was masked from further analysis. 


Original AVIRIS imagery courtesy of NASA. 


analysis based on a dual inspection of the spatial co- 
herency of the eigenimages and the corresponding e1- 
genvalue plot. Generally, the more spatially coherent 
the image, the less noise and greater the information 
content. A graph of the eigenvalues by band reveals 
that the first 10 eigenimages contain most of the valu- 
able information. MNF eigenimages with values close 
to 1 contain mostly noise. MNF bands with eigenval- 
ues >1 account for most of the variance in the hyper- 
spectral dataset. 


Care should be exercised in the selection of the MNF 
bands to keep for subsequent hyperspectral image pro- 
cessing. For example, only the first 19 MNF bands as- 
sociated with run 08 scene 05 were deemed useful for 
subsequent hyperspectral endmember analysis of the 
Savannah River Site study area, though a smaller sub- 
set could be justified. The number of MNF transform 
eigenimages passed to subsequent analytical steps (and 
hence, the inherent dimensionality of the dataset) will 
determine the number of endmembers, or unique ma- 
terials that can be defined (see next section). 


The aforementioned example was based on an entire 
scene of AVIRIS data. It is often useful to mask and/or 
subset the scene according to the most important fea- 
tures of interest. For example, consider Figure 11-10 
where only the clay-capped hazardous waste sites asso- 
ciated with the Mixed Waste Management Facility are 
retained after masking. Figure 11-10a is a masked 
false-color composite of three AVIRIS bands (RGB = 
50, 30, 20). Figure 11-10b is a masked false-color com- 
posite of MNF bands (RGB = 3, 2, 1). A binary mask 
was applied to the image such that features not of in- 
terest, including null data values, were excluded from 
further analysis. The MNF transform was then applied 
to the masked dataset, and the result was passed to the 
next step in the hyperspectral image processing chain. 


Endmember Determination: 
Locating the Spectrally 
Purest Pixels 


The primary goal of most hyperspectral analyses is to 
remotely identify the physical or chemical properties of 
materials found within the IFOV of the sensor system. 
Spectral mixture analysis (SMA) and its variants are 
powerful algorithms that can be used to determine the 
relative proportions of pure endmember spectra in a 
pixel. Endmember spectra are target land-cover classes 
believed to be present in the data that are unmixed or 
pure (Roth et al., 2012). Potential endmember spectra 
can be collected from spectroradiometer measurements 
collected in the lab, in the field, simulated using radia- 
tive transfer modeling and/or directly from the satellite 
or airborne remote sensor data (Roth et al., 2012). 


Most of the time the spectral reflectance entering the 
IFOV of the sensor system at the instant of data collec- 
tion is a function of the radiant flux from a variety of 
endmember materials. If we can identify the spectral 
characteristics of the endmember materials, then it 
may be possible to identify pixels that contain varying 
proportions of these materials. There are several meth- 
ods of identifying the most spectrally pure pixels in a 
multispectral or hyperspectral scene, including: 


* pixel purity index, and 
¢ n-dimensional visualization of endmembers in fea- 
ture space. 


In the Savannah River Site study, the goal was to deter- 
mine if there were any vegetation stress endmembers 
associated with Bahia or Centipede grass growing on 
flat clay-capped hazardous waste sites. 
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Pixel Purity Index Images 


a. Original. 


b. Threshold at 10 hits highlighted in red. 


FIGURE 11-11 Pixel purity index (PPI) image of a portion of run 08, scene 05 derived from masked minimum noise fraction 
AVIRIS data. The pixels in (a) denote the original output from the PPI analysis. The pixels highl ight ed in red in (b) represent a 
subset of those pixels selected using a threshold of 10 hits. Original AVIRIS imagery courtesy of NASA. 


Pixel Purity Index Mapping 


It is sometimes difficult to identify pure endmembers 
in a remote sensing dataset because very few pixels 
contain just one type of biophysical material. There- 
fore, a rigorous mathematical method of determining 
the most spectrally pure pixels is to repeatedly project 
n-dimensional scatterplots of the clean minimum noise 
fraction images (e.g., not just 3 bands as in the previ- 
ous example but a// useful MNF bands) onto a ran- 
dom unit vector. Each time the spectral data are 
projected, the most extreme pixels in each projection 
are noted (Research Systems, Inc., 2004; Exelis ENVI, 
2014). By keeping track of which pixels in the scene are 
repeatedly identified as “extreme” pixels, it is possible 
to create a pixel purity index (PPI) image. Basically, the 
higher the pixel value in the pixel purity index image, 
the greater the number of times it was judged to be a 
spectrally extreme pixel (e.g., a pure water, pure con- 
crete, or pure vegetation endmember). It is important 
to include as input to the PPI calculation only MNF 
images that contain spectrally valuable information. 


A pixel purity index image derived from an analysis of 
masked minimum-noise-fraction images is shown in 
Figure 11-lla. Note that there are not a tremendous 
number of relatively pure pixels in the scene. In fact, 
only 4,155 pixels in the masked clay-cap image were 
identified as being pure to some degree. Furthermore, 
a threshold is typically used to create a subset of the 
most pure PPI pixels to be used in subsequent analyses 
(Figure 11-11b). In this case, only 1,207 of the 4,155 
pixels were selected for further processing after 70,000 
PPI iterations using a conservative threshold value of 
10 “hits.” 


A PPI image simply identifies the location of the most 
pure pixels (i.e., endmember candidates). Unfortunate- 
ly, it is difficult to label the types of endmembers by 


just viewing the two-dimensional pixel purity index im- 
age. Therefore, it is common to use an n-dimensional 
visualization technique. 


n-Dimensional Endmember 
Visualization 


To identify and label the most spectrally pure endmem- 
bers in the two-dimensional pixel purity index image, it 
is necessary to systematically view where the relatively 
few pure vegetation pixels reside in n-dimensional fea- 
ture space or in the image itself. For example, Figure 
11-12 depicts a display of the most spectrally pure pix- 
els found in the PPI image shown in Figure 11-10b. By 
using more than two of the MNF bands at a time it is 
possible to interactively view and rotate the endmem- 
bers in n-dimensional spectral space on the computer 
screen. The analyst attempts to locate the corners of 
the data cloud in n-dimensional space as the cloud ro- 
tates. The purest pixels lie at the convex corners of the 
data cloud. Clusters can be defined manually, or an al- 
gorithm can be used to precluster the data, and the 
endmembers can be subsequently refined manually. In 
this context, m equals the number of MNF bands, and 
given the dimensionality of the simplex formed by the 
cluster means, the number of endmembers that can be 
defined is equal to n + 1. Thus, there are theoretically 
n+ 1 endmembers if the data are truly n-dimensional 
(Boardman, 1993). 


By comparing the actual spectra of an endmember 
found within the n-dimensional display with where it is 
actually located in x, y space in the MNF or surface re- 
flectance image, it is possible to label the pure pixel as a 
specific type of endmember, e.g., water, vegetation, or 
bare soil. Endmembers that are extracted in this man- 
ner may then be used to perform spectral matching or 
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Three-Dimensional Visualization of Five Endmembers 


FIGURE 11-12 N-dimensional visualization of four potential image-derived vegetation endmembers and one shade end- 
member derived from the pixel purity index (PPI) analysis. Endmembers associated with other materials (e.g., asphalt, con- 
crete) are not highlighted. Note that some of the highlighted pixels are not immediately recognizable as endm embers. Data 
cloud rotations were necessary to discern all the endmember regions of interest. 


classification. More automated methods of endmem- 
ber determination are available. 


To demonstrate these methods, let us examine the clay- 
capped hazardous waste sites in the run 08 scene 05 da- 
taset to identify and label the following spectra: 


¢ healthy Bahia grass, and 
* potentially stressed Bahia grass. 


Masked MNF images were analyzed to produce the 
pixel purity index image (and spectral profiles of the 
most pure pixels) previously discussed (Figure 11-9). 
These data were then displayed as data clouds in an n- 
dimensional feature space (Figure 11-12). Spectra 
(scaled reflectance values) for one healthy vegetation 
endmember (37) and three potentially stressed vegeta- 
tion endmembers (15, 25, and 36) over the wavelength 
interval from 0.4 to 2.43 Km are superimposed in Fig- 
ure 11-13a. The same spectra are replotted in the visi- 
ble through near-infrared region (0.4-0.9 Xm) in 
Figure 11-13b to highlight certain reflectance charac- 
teristics. The numerical names assigned to these end- 
member spectra (e.g., #37) are an artifact of the 
iterative endmember selection process. 


The clay-cap on the Mixed Waste Management Facili- 
ty (MWMF) contains a near-monoculture of Bahia 
grass (there are a few patches of Centipede). Bahia 
grass endmember 37 exhibited all the characteristics of 
healthy vegetation, including a) strong chlorophyll a 
and 6 absorption bands in the blue (0.43 — 0.45 Xm) 
and red (0.65 — 0.66 Xm) portions of the spectrum, b) a 
relative lack of absorption in the wavelength interval 
between the two chlorophyll absorption bands, pro- 
ducing an increase in reflectance at approximately 0.54 
Xm in the green portion of the spectrum, c) a substan- 
tial increase in reflectance throughout the near-infra- 
red portion of the spectrum (0.7 — 1.2 Xm) due to 
scattering and reflectance in the spongy mesophyll por- 
tion of the plant, and d) typical absorption/reflectance 
characteristics in the middle-infrared (1.45 — 1.79 Km 
and 2.08 — 2.35 Xm) wavelengths (Jensen, 2007). 


Conversely, each potentially stressed Bahia grass end- 
member (15, 25, and 36) exhibited spectral reflectance 
characteristics that were very different from the healthy 
Bahia grass spectra. Relative to the healthy turfgrass 
spectrum, each exhibited a) a slight to moderate in- 
crease in red reflectance (endmember 15 most; 25 inter- 
mediate; 36 least); b) a slight to moderate red-edge 
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Healthy and Potentially Stressed Bahia Grass 
AVIRIS Endmember Spectra 
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Fl GURE 11-13 a) One healthy (37) and three potentially stressed Bahia grass image-derived endmember spectral profiles 
derived using a pixel purity index image and n-dimensional visualization endmember analysis. b) Only the visible through 
near-infrared region (0.4-0.90 Kim) is plotted to highlight the increase in red reflectance, the apparent red-edge shift, and de- 
crease in near-infrared reflectance of the potentially stressed vegetation endmembers. 


shift toward shorter wavelengths in the transition re- 
gion from red to near-infrared wavelengths (15 most; 
25 intermediate; 36 least), and c) a significant decrease 
in near-infrared reflectance. 


Endmember 15 consistently reflected more energy in 
both middle-infrared regions (1.45 — 1.79 Kim and 2.08 
— 2.35 Mm) than the healthy Bahia grass endmember 


(37). Endmember 25 exhibited a substantial decrease 
in middle-infrared reflectance in the 1.45 — 1.79 Xm re- 
gion and approximately the same reflectance as the 
healthy vegetation in the 2.08 — 2.35 Km region. Final- 
ly, endmember 36 exhibited a decrease in middle-infra- 
red reflectance in the 1.45 — 1.79 Xm region and a slight 
increase in reflectance in the 2.08 — 2.35 Xm region. 
While the identified endmembers are spectrally pure in 
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Selected Healthy and Stressed Bahia Grass AVIRIS Spectra 
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FIGURE 11-14 a) Spectra located within areas of homogeneous healthy and stressed Bahia grass. b) The visible through 
near-infrared region is rescaled to highlight the increase in red reflectance, the red-edge shift, and decrease in near-infrared 


reflectance of the potentially stressed vegetation endmembers. 


theory, it is possible that albedo contributions from 
other materials are present in the pixels (e.g., soil back- 
ground reflectance). However, the extracted pixels are 
the purest relative to all others in the image. 


To illustrate the utility of the endmember concept fur- 
ther, consider the reflectance profiles obtained at two 
randomly selected healthy and potentially stressed ar- 
eas on the Bahia grass clay-cap (Figure 11-14ab). Note 
that these non-endmember profiles retain the same 
general relationships as the image-derived endmember 
profiles (Figure 11-13ab). 


Before proceeding, one should ask, “How can we be 
sure that the Bahia grass is actually undergoing some 
stress?” Fortunately, we have some laboratory Bahia 


grass spectra that depict how healthy and stressed Ba- 
hia grass reflect electromagnetic energy in a controlled 
environment. For example, Figure 11-15 depicts the 
spectra of healthy Bahia grass and Bahia grass subject- 
ed to various concentrations of copper (mg/ml) 
(Schuerger, 2003). Note the increase in reflectance in 
the red region, the red-edge shift, and the decrease in 
reflectance in the near-infrared region as the copper 
concentration is increased. Unfortunately, the labora- 
tory spectroradiometer did not record Bahia grass 
spectral information in the middle-infrared region. 
Thus, the AVIRIS hyperspectral results only suggest 
that the spectra labeled “potentially stressed Bahia 
grass” are indicative of some type of stressing agent. It 
is not suggested that the stress on the MWMF is due to 
the absorption of copper, but rather that the stressed 
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Laboratory-Derived Bahia Grass Spectra 
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FIGURE 11-15 Laboratory spectral reflectance of healthy and copper-stressed Bahia grass in the region from 0.4 to 0.9 Xm 


(Schuerger, unpublished data). 


Bahia grass spectra simply exhibit stressed behavior 
throughout the visible and near-infrared portions of 
the spectrum (it could be completely different in the 
middle-infrared region). The agent responsible for the 
potential vegetation stress may be a) an overabundance 
of water, resulting in a decrease in middle-infrared re- 
flectance, as with endmembers 25 and 36; b) too little 
water, resulting in an increase in reflectance in the mid- 
dle-infrared region, as with endmember 15; or c) a 
completely different stressing agent related to clay-cap 
constituents. 


Mapping and Matching using 
Hyperspectral Data 


Algorithms may be used to convert hyperspectral re- 
flectance into biophysical information or thematic 
maps, including the spectral angle mapper (SAM), 
subpixel classification (e.g., linear spectral unmixing), 
spectroscopic library matching, hyperspectral indices, 
and derivative spectroscopy. Scientists have also ap- 
plied traditional classification algorithms and machine 
learning techniques to hyperspectral data (e.g., Goel et 
al., 2003; Im et al., 2009, 2012). 


Spectral Angle Mapper 


Spectral angle mapping (SAM) algorithms take an at- 
mospherically corrected unlabeled pixel (e.g., com- 
posed of an AVIRIS measurement vector of n 


brightness values) and compare it with reference spec- 
tra in the same n-dimensions. In the SRS study, only 
175 of the original 224 AVIRIS bands were used. The 
reference spectra may be obtained using 1) calibrated 
in situ or laboratory-derived spectroradiometer mea- 
surements stored in an ASCII or binary spectral li- 
brary, 2) theoretical calculations, and/or 3) 
multispectral or hyperspectral image endmember anal- 
ysis procedures, as previously discussed. The SAM al- 
gorithm compares the angle (a ) between the reference 
spectrum (r) and the hyperspectral image pixel mea- 
surement vector (ft) in m-dimensions (bands) (Figure 
11-16) and assigns it to the reference spectrum class 
that yields the smallest angle (Research Systems, Inc., 
2004; Belluco et al., 2006; Exelis ENVI, 2014): 


nN 

> tr, 
| i=1 

a = cos (11.1) 


n 2 n 
eau cal 


i=1 i=1 


The simple two-band example in Figure 11-16 suggests 
that unknown material ¢ has reflectance spectra that 
are more similar to reference spectrum r than the spec- 
trum for material k, 1.e., the angle (a) in radians be- 
tween r and ¢ is smaller. Basically, for each reference 
spectrum r (this could be an in situ-derived endmem- 
ber or a remote sensing—derived endmember, as with 


480 INTRODUCTORY DIGITAL IMAGE PROCESSING 


Spectral Angle Mapper Logic 
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FIGURE 11-16 The logic associated with the Spectral Angle 
Mapper (SAM). 


the four endmembers previously discussed) a spectral 
angle (a ) is computed for each unknown image spec- 
trum (pixel) in the dataset. This angle becomes the pix- 
el value in the output SAM image with one output 
image for each reference spectrum. This creates a new 
SAM datacube with the number of bands equal to the 
number of reference spectra used in the mapping. 


The SAM algorithm was applied to the previously de- 
scribed endmembers. Thematic maps of the spatial dis- 
tribution of healthy (derived from endmember 37) and 
potentially stressed Bahia grass (derived from end- 
members 15, 25, and 36) are shown in Figure 11-17. 
The darker the pixel (1.e., black, blue) the smaller the 
SAM angle and the closer the match. Thus, pixels in 
Figure 11-17a that had spectra very similar to that of 
the healthy Bahia grass endmember spectra (37) yield- 
ed very small angles and were color-coded as being 
black or dark blue. Note that the spatial distribution of 
the healthy vegetation derived from endmember 37 is 
almost the inverse of the maps derived from the poten- 
tial stress endmembers (15, 25, and 36) in Figure 11- 
17b-d. 


It is possible to convert the output from the spectral 
angle mapper subroutine into a hard (crisp) classifica- 
tion map. For example, Figure 11-18a depicts a hard 
classification map derived using just the four endmem- 
bers previously discussed, one class for each endmem- 
ber. A threshold angle value of 0.1 radian was used. 
Brown areas represent healthy vegetation (based on 
endmember 37) and the other three classes represent 
potentially stressed vegetation. 


The clay-capped hazardous waste sites ideally have a 
relatively homogenous cover of Bahia grass. Maps 
such as this suggest that on this date the Bahia grass 
may not have been uniformly distributed and that cer- 
tain locations within the site may have stressed Bahia 
grass. This stress could be the result of a number of 
factors. The display provides valuable spatial informa- 
tion that can be of significant value to the personnel re- 
sponsible for clay-cap maintenance. It can help them to 
focus their attention on certain locations on the clay- 
cap to ensure its continued integrity. 


It is also possible to use the laboratory-derived Bahia 
grass reference spectra as endmember spectra to be in- 
put to SAM. A potentially stressed Bahia grass classifi- 
cation map derived from averaged laboratory spectra 
with a 0.5 mg/ml copper treatment is shown in Figure 
11-18b. The darker the pixel (blue, black, etc.), the 
smaller the angle and the closer the match. It visually 
correlates well with the results obtained from the re- 
mote sensing—derived spectra. 


Subpixel Classification, Linear 
Spectral Unmixing or Spectral 
Mixture Analysis 


The energy recorded by a remote sensing detector is a 
function of the amount of energy reflected or emitted 
by the materials within the instantaneous-field-of-view 
(IFOV). For example, consider the single pixel shown 
in Figure 11-19a, which is composed of 50% water, 
25% bare soil, and 25% vegetation. Should this pixel be 
classified as water because its dominant constituent is 
water? Wouldn’t it be more useful to determine and re- 
port the actual proportion (or abundance) of the pure 
endmember (class) materials within the pixel? This pro- 
cess is commonly referred to as subpixel classification, 
linear spectral unmixing, or spectral mixture analysis 
(Okin et al., 2001; Jensen et al., 2009; Somers et al., 
2011; Thenkabail et al., 2011; Prost, 2013). 


Spectral mixture analysis (SMA) can be used to de- 
compose the measured spectral reflectance within the 
IFOV of a single pixel as a mixture of a fixed set of 
endmembers. The mixing can be modeled as the sum of 
endmember reflectance multiplied by their fraction 
contribution to a best-fit mixed spectrum. Linear SMA 
is based on the equation (Roth et al., 2012): 


nN 


P= DY) in Xf) te, 

i=1 
where p', is the reflectance at wavelength, 4; p,, is 
the spectral reflectance of endmember i, and /; is the 
fractional cover of endmember i. The total number of 
endmembers in the model is m and the model error is 


(11.2) 
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ClassiXlcation Maps of Healthy and Potentially Stressed Vegetation 
Created Using Image-Derived Endmembers and a Spectral Angle Mapper Algorithm 


c. Potential stress: derived from endmember 25. d. Potential stress: derived from endmember 36. 


FIGURE 11-17 a) Healthy Bahia grass classification map derived from July 26, 1999, AVIRIS hyperspectral data and spectral 
angle mapper (SAM) analysis of endmember 37. b—d) Potentially stressed Bahia grass classification map derived from AVIRIS 
hyperspectral data and SAM analysis of endmembers 15, 25, and 36. In all cases, the darker the pixel (blue, black, etc.), the 
smaller the angle and the closer the match. Original AVIRIS imagery courtesy of NASA. 


ClassiXcation Map of Potentially Stressed 
Vegetation Created Using Laboratory—Derived 
Hardened ClassiXcation Map of Healthy Endmembers and the Spectral 
and Potentially Stressed Vegetation Angle Mapper Algorithm 


a. Hardened Spectral Angle Mapper classiXcation map. b. Laboratory-derived endmember classiXcation map. 


Fl RE 11-18 a) A hardened classification map derived from July 26, 1999, AVIRIS hyperspectral data and spectral angle 
mapper (SAM) analysis of four endmembers. Brown areas represent healthy Bahia grass. White, red, and yellow depict areas 
with vegetation stress-related characteristics. b) Potentially stressed Bahia grass classification map derived from laboratory 
spectra with a 0.5 mg/ml copper treatment. The darker the pixel (blue, black, etc.), the smaller the angle and the closer the 
match. Original AVIRIS imagery courtesy of NASA. 
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Linear Mixing Model for a Single Pixel 
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Fl GURE 11-19 a) Linear mixture model ing associated with a single pixel consisting of water, vegetation, and bare soil. 
b) The location of hypothetical mean vectors and endmembers in two-dimensional feature space (red and near-infrared). 


é,. The accuracy of the SMA model fit is determined 
by computing the root mean square error (RMSE) 
(Roth et al., 2012): 


(11.3) 


where b is a band number and k is the total number of 
bands. 


To appreciate linear spectral unmixing, let us assume 
for a moment that we have a remotely sensed image 
consisting of just two bands (A = 2) and that there are 
only three types of pure endmember materials in the 
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scene (n = 3): water, vegetation, and bare soil—p,,, 
P5,, and p3,, respectively. The linear mixing associat- 
ed with any pixel in the scene can be described using 
the matrix notation: 


[Pa] = [alll 


where [p', ] is the k-dimensional spectral vector of the 
pixel under investigation, Yd is the n X 1 vector of n 
endmember fractions (e.g., 0.5, 0.25, 0.25) for the pixel 
and [p,] is the & x n signature matrix, with each col- 
umn containing one of the endmember spectral vectors 
(Schowengerdt, 2007). The relationship contains some 
noise, therefore ¢ represents the residual error. This 
equation assumes that we have identified all of the the- 
oretically pure classes (endmembers) in the scene such 
that their proportions (f;) will sum to | at each pixel 
and that all endmember fractions are positive. 


(11.4) 


If we assume that the data contain no noise (€ ), Equa- 
tion 11.4 becomes: 


Pa) = [al 


and we can use the following matrix notation to solve 
for [ f| on a pixel-by-pixel basis: 


(11.5) 


' ater 
PS = Pwater3 Preg3 Pbsoil3 f (11.6) 
' veg 
P4 Pwater4 Preg4 Phsoil4 Fi 
bsoil 


Unfortunately, this relationship is under-determined 
because there are fewer bands than endmembers. For- 
tunately, because we know that the linear combination 
of all three endmembers must sum to 1 (1.¢., fiyater + 
vegetation + Share soil = 1), we can augment the equation 
to become: 


p'3 P water3 Preg3 Phsoil3 water 
‘| = 11.7 
P4 Pwater4 Preg4 Phsoil4 Sveg ( ) 
I I 1 1 JL iail 


This equation can be inverted to solve for the exact 
fraction (proportion) of endmember materials found 
within each pixel: 


1 


water Pwater3 Preg3 Phsoil3 P'3 
= ' 11.8 
Srveg Pwater4 Preg4 Posoil4 P4 ( ) 
J aati 1 1 1 1 
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TABLE 11-2 Reflectance values for the three hypothetical 
image-derived endmembers shown in Figure 11-19b. 


Band Water Vegetation Bare Soil 
3 13 22 70 
4 5 80 60 


—l 
Please note that lp | is now lp | 


Now let us use Equation 11.5 to determine the propor- 
tion of endmember materials in pixels in a hypothetical 
scene. Consider the two-dimensional feature space plot 
shown in Figure 11-19b. This scene consists of just two 
bands of remote sensor data (band 3 and 4; red and 
near-infrared, respectively). The yellow cloud repre- 
sents the co-occurrence of all pixel values in the scene 
in bands 3 and 4. There are no pixel values in the scene 
outside the yellow area convex hull. The circles are 
mean vectors associated with water, vegetation, and 
bare soil obtained through supervised training. They 
are present to help the reader appreciate the difference 
between mean vectors and endmembers and are not 
used in this computation. The three stars in the display 
represent pure endmember pixels associated with wa- 
ter, vegetation, and bare soil. These endmembers were 
derived from the image itself using pixel purity index 
assessment and/or n-dimensional visualization or by 
obtaining accurate spectroradiometer measurements 
on the ground of pure water, vegetation, and bare soil. 


This graphic contains enough data for us to derive the 
information necessary to predict the proportion of 
each endmember in each pixel in the scene. The values 
found in Figure 11-19b are summarized in Table 11-2. 
It is possible to compute the fractions of each end- 
member class found in each pixel in the scene by a) 
placing the inverse matrix [p, coefficients in Table 
11-3 into Equation 11-5, and b) repeatedly placing new 
values of p,,; and p,, in Equation 11.5 associated 
with each pixel in the scene. For example, if the p, 
and p,, values for a single pixel were 25 and 57, re- 
spectively, then the proportion (abundance) of water, 
vegetation, and bare soil found within this single pixel 
would be 27%, 61%, and 11%, as shown below: 


0.0053 -0.0127 1.1322 | [25 
Syeg | ~ |-0.0145 0.0150 0.1137 | |57 


D pitler 


Fosoit| — {.9-0198 -0.0024 -0.2460] 1 | 
0.27| [0.0053 —0.0127 1.1322 ] [25] 
0.61| = |-0.0145 0.0150 0.1137||57 


0.11] | 0.0198 —0.0024 —0.2460) | 1 
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TABLE 11-3 Matrices used to perform spectral unmix- 
ing.The inverse matrix [p, |” was derived from the end- 
member reflectance values in [p, ]. 


[P| P] 


13 22 70 = 0:00535— 00127 Mls 22, 
5 80 60 —0.0145 0.0150 0.1137 
i i i 0.0198 —0.0024 —0.2460 


This logic may be extended to hyperspectral imagery 
and the overdetermined problem, where there are more 
bands than endmembers. Schowengerdt (2007) de- 
scribes the more complex pseudo-matrix inversion that 
must be adopted when analyzing hyperspectral data. 


Spectral mixture analysis (SMA) assumes a pixel’s 
spectrum is a linear combination of a finite number of 
spectrally distinct endmembers. Spectral mixture anal- 
ysis uses the dimensionality of hyperspectral data to 
produce a suite of abundance (fraction) images for 
each endmember. Each fraction image depicts a sub- 
pixel estimate of endmember relative abundance as 
well as the spatial distribution of the endmember. 
When the endmembers include vegetation, the end- 
member fraction is proportional to the areal abun- 
dance of projected canopy cover (Roberts et al., 1993; 
Williams and Hunt, 2002). Although SMA is intuitive- 
ly very appealing, scientists often find it difficult to 
identify all of the pure endmembers in a scene (Jensen 
et al., 2009). 


McGwire et al. (2000) found that endmembers (average 
green leaf, soil, and shadow) derived from hyperspec- 
tral data in arid environments were more highly corre- 
lated with vegetation percent cover than when the 
hyperspectral data are processed using the traditional 
narrow-band and broadband vegetation indices (e.g., 
NDVI, SAVI, EVI). Williams and Hunt (2002) used 
mixture-tuned matched filtering (MTMF), a special 
case of subpixel analysis, to estimate leafy spurge cano- 
py cover and map its distribution. Segl et al. (2003) 
used endmembers derived from hyperspectral data and 
linear spectral unmixing to identify urban surface cov- 
er types. Franke et al. (2009) used hierarchical multiple 
endmember spectral mixture analysis (MESMA) to 
map urban land cover. Hatala et al. (2010) used 
MTMF to identify forest pest and pathogen damage in 
the Greater Yellowstone Ecosystem. Roth et al. (2012) 
compared several endmember selection techniques and 
evaluated the tradeoffs between classification accuracy 
achieved, the size of the library created, and computa- 
tion time. 


Continuum Removal 


Even when great care is given to the selection of end- 
members during SMA, the classification results may 
not be optimal because of the spectral similarity be- 
tween classes, especially for similar plant types or 
closely-related plant species. Youngentob et al. (2011) 
suggest that continnum removal (CR) analysis may im- 
prove class separability by emphasizing individual ab- 
sorption features across a normalized spectrum. CM 
involves fitting a spectral reflectance curve with a con- 
vex hull (i.e., a line just touching the maximum points 
along the original reflectance curve). The continuum is 
removed by dividing reflectance values at a specific 
wavelength (p, ) by the reflectance value of the contin- 
num (p,,) at the corresponding wavelength 2. The 
continuum removed (CR) spectral reflectance (p,... ) at 
wavelength ~ is given by (Youngentob et al., 2011; 
Rodger et al., 2012): 


es 


(11.9) 
Por 


Porn ~ 


Applying continuum removal to the scaled reflectance 
data enhances the subtle absorption features in the 
spectral reflectance curves because their depth and po- 
sition are not influenced by variations in albedo (Filip- 
pi and Jensen, 2007). For example, consider the 
reflectance curve of healthy vegetation sampled every 
50 nm from 400 to 2,500 nm in Figure 11-20a and its 
associated continuum (convex hull). The application of 
Equation 11.9 produces a new vegetation reflectance 
curve shown in Figure 11-20b. The difference between 
the green reflectance maximum and the red reflectance 
minimum in the original reflectance curve is approxi- 
mately 5—7% (Figure 11-20a). The difference between 
the green maximum and the red minimum reflectance 
is approximately 20% in the continuum-removed curve 
(Figure 11-20b). The emphasis of the red absorption 
feature (and other absorption features not discussed) 
in the continuum-removed data may make it possible 
to extract more accurate information when the data are 
analyzed using SMA or other techniques (e.g., NDVI). 


Spectroscopic Library Matching 
Techniques 


One of the benefits of conducting hyperspectral remote 
sensing research is that a detailed spectral response cal- 
ibrated to percent reflectance (spectra) can be obtained 
for a pixel (e.g., AVIRIS measures in 224 bands from 
400 to 2500 nm). The remote sensing—derived spectra 
can be compared with in situ- or laboratory-derived 
spectra that is stored in a special spectral library. The 
laboratory-derived spectra are usually considered more 
accurate because they have been made under con- 
trolled illumination conditions, and the atmosphere is 
not a factor. Laboratory-derived spectra may be found 
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FIGURE 11-20 a) Spectral reflectance curve of healthy green vegetation sampled every 50 nm from 400 to 2,500 nm fitted 
with a continuum (convex hull) that connects the points of maximum reflection with a straight line. b) Vegetation reflectance 


after the continuum has been removed using Equation 11.9. 


at the ASTER Spectral Library (2014), which contains 
the: 


¢ Johns Hopkins Spectral Library, 


¢ NASA Jet Propulsion Laboratory Spectral Library, 
and 


¢ U.S. Geological Survey Spectral Library. 


Several digital image processing software vendors pro- 
vide library spectra as an appendix to their software 
(e.g., ERDAS, ENVI, PCI Geomatica). 


Absorption features are present in the remote sensing— 
or laboratory-derived reflectance spectra (seen as local- 
ized dips) due to the existence of specific minerals, 
chlorophyll a and 6, water, and/or other materials with- 
in the pixel IFOV. The absorption features in the spec- 
tra are characterized by their spectral locations (i.e., 
the bands affected), their depths, and widths. 


Considerable attention has been given to developing li- 
brary matching techniques that allow remote sensing— 
derived spectra to be compared with spectra that were 
previously collected in the field or in the laboratory. 
For example, we may have remote sensing—derived 
spectra from an agricultural field and desire to com- 
pare it with a library containing a significant variety of 
agricultural spectra. We would compare each pixel’s 
spectrum with each of the agricultural spectra in the li- 
brary and assign the pixel to the class its spectra most 
closely resembles. 


Comparing the remote sensing—derived spectra with 
the stored laboratory-derived spectra is no small mat- 


ter. First, if we are using AVIRIS data, then each pixel 
has the potential of being represented by 224 distinct 
spectral band measurements. Each of these distinct 
measurements would have to be compared with each of 
the related bands in the library reference spectra. The 
library may contain hundreds of spectra and the re- 
mote sensing scene often consists of millions of pixels. 
The computation required to conduct true band-to- 
band comparison between remote sensing—derived 
spectra and library spectra is daunting. Therefore, var- 
ious coding techniques have been developed to repre- 
sent an entire pixel spectrum in a simple yet effective 
manner so that it can be efficiently compared with li- 
brary spectra. 


One simple coding technique is binary spectral encod- 
ing (Jia and Richards, 1993). Binary spectral encoding 
can be used to transform a hyperspectral reflectance 
spectrum into simple binary information. The algo- 
rithm is: 

b(k) = 0 


if p(k)<Ty, (11.10) 


otherwise 
b(k) = 1 


where p(x) is the reflectance value of a pixel in the xth 
spectral band, 7) is a user-selected threshold for creat- 
ing the binary code, and h(x) is the output binary code 
symbol for the pixel in the k'” spectral band. T, may be 
the average reflectance value of the entire spectrum for 
a pixel or a user-specified value corresponding to a 
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b. Binary Codes f. Hamming Code 
Lab In Situ RS. RS. Lab In Situ R.S. Hamming 
nm Vegetation Vegetation Water Vegetation Vegetation Code 
500 0 0 0 0 0 0 
520 0 0 0 0 0 0 
540 1 1 0 1 1 0 
560 1 1 0 1 1 0 
580 1 0 0 1 0 1 
600 0 0 0 0 0 0 
620 0 0 0 0 0 0 
640 0 0 0 0 0 0 
660 0 0 0 Q versus 0) 0 
680 0 0 0 0 0 0 
700 1 0 0 1 0 1 
720 1 1 0 1 1 0 
740 1 1 0 1 1 0 
760 1 1 0 1 1 0 
780 1 1 0 1 1 0 
800 1 1 0 1 1 0 
g. Hamming Code h. Hamming Code 
Lab Jn Situ R.S. Hamming R.S. RS. Hamming 
nm Vegetation Water Code Vegetation Water Code 
500 0 0 0 0 0 0 
520 0 0 0 0 0 0 
540 1 0 1 1 0 1 
560 1 0 1 1 0 1 
580 1 0 1 0 0 0 
600 0 0 0 0 0 0 
620 0 0 0 0 0 0 
640 0 0 0 0 0 0 
660 0 versus 0 0 Q versus (0 0 
680 0 0 0 0 0 0 
700 1 0 1 0 0 0 
720 1 0 1 1 0 1 
740 1 0 1 1 0 1 
760 1 0 1 1 0 1 
780 1 0 1 1 0 1 
800 1 0 1 1 0 1 


Hamming code distance (number of times binary digits are different): 

Lab in situ vegetation versus remote sensing—derived vegetation = 2 

Lab in situ vegetation versus remote sensing—derived water = 9 

Remote sensing—derived vegetation versus remote sensing—derived water = 7 


Fl @JRE 11-21 a) In situ vegetation spectrum obtained using a spectroradiometer in a controlled laboratory environment 
(green) and vegetation and water spectra obtained from a remote sensing instrument (red). b) Binary encoding of the in situ 
vegetation, remotely sensed vegetation, and water data. c) Graph of the binary encoding of the in situ vegetation data. 

d) Graph of the binary encoding of the remote sensing—derived vegetation data. e) Graph of the binary encoding of the 
remote sensing-derived water data. f) Hamming code used in the computation of the Hamming distance between in situ 
vegetation and remote sensing-derived vegetation. g) Hamming code used in the computation of the Hamming distance 
between in situ vegetation and remote sensing—derived water. h) Hamming code used in the computation of the Hamming 
distance between remote sensing—derived vegetation and water. 


unique absorption spectra of particular interest. For * vegetation and water spectra obtained using an air- 
example, consider Figure 11-21a, which displays: borne hyperspectral sensor. 


* an in situ vegetation reflectance spectrum measured 
in a laboratory using a spectroradiometer, and 


A threshold of 10% reflectance was chosen because it 
captured nicely the chlorophyll a and b absorption 
bands in the blue and red portions of the spectrum. 
With the threshold set at 10%, it was a straightforward 
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task to binary encode each of the three spectra and 
then compare their binary codewords to see how simi- 
lar or different they were. For example, the in situ vege- 
tation spectra exceeded the threshold in the region 
from 540 to 580 nm and again in the region from 700 
to 800 nm. Therefore, each of these wavelength regions 
(bands) received a value of 1 in their codewords in Fig- 
ure 11-21b. The remote sensing—derived vegetation 
spectra had very similar characteristics, exceeding the 
threshold from 540 to 560 nm and from 720 to 800 nm. 
Once again, all these bands were coded to 1. The re- 
mote sensing—derived water spectra never exceeded the 
threshold and therefore received zeros in all bands. 


Thus, each of the original spectra was turned into a 16- 
digit codeword, cj, cp, and c3. But how do we use these 
three binary codewords (each of length Z = 16) to de- 
termine if the binary spectra of one class is identical, 
similar to, or completely different from the spectra of 
another class? Two spectra that have been binary en- 
coded can be compared by computing the Hamming 
distance between their binary codewords using: 


N = bands 


Dist pram Cp c) = > 
k=1 


[e()OcKk)] (11.11) 


where c; and c; are two spectral codewords of length L 
and the © symbol represents exclusive OR Boolean 
logic. The algorithm compares each bit of the two 
codewords under examination and outputs a 0 where 
they agree and a | where they are different. The result 


is a Hamming distance codeword of length Z. 


Thus, the Hamming distance is computed by summing 
the number of times the binary digits are different. For 
example, consider the computation of the Hamming 
distance between the laboratory in situ vegetation 
codeword and the remote sensing—derived vegetation 
codeword. The two codewords differ at only two loca- 
tions in the 16-bit lengths and therefore have a Ham- 
ming distance of 2. Conversely, when the laboratory in 
situ vegetation codeword is compared with the water 
codeword, it differs at nine locations, resulting in a 
Hamming distance of 9. 


The more similar one spectra is to another, the lower 
its Hamming distance. Thus, one can a) identify the 
appropriate threshold for a material or land-cover class 
of interest in light of existing absorption bands, b) 
compute a binary codeword at each pixel, and c) com- 
pare this binary codeword at each pixel with m library 
spectra that have been binary coded in a similar man- 
ner. Each pixel could be assigned to the class that re- 
sults in the smallest Hamming distance. 
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Library Matching Using Binary Encoding 
at Selected Locations 
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FIGURE 11-22 The location of three thresholds that could 
be used to perform binary encoding of the three spectra 
from 540 to 560 nm, 660 to 680 nm, and 740 to 760 nm set 
at 10%, 5%, and 40% reflectance, respectively. 


Note that more than one threshold can be used a) 
within a chosen absorption band, and b) at numerous 
disjoint locations along the spectra. For example, Fig- 
ure 11-22 depicts the location of three unique thresh- 
olds that could be used to perform binary encoding 
using just the information at 7, (540 to 560 nm), 7 
(660 to 680 nm), and 73 (740 to 760 nm) set at 10%, 
5%, and 40% scaled reflectance, respectively. 


Machine Learning Analysis of 
Hyperspectral Data 


Scientists have also used various types of machine- 
learning algorithms to analyze hyperspectral data such 
as decision trees and support vector machines (SVM). 


Decision Tree Analysis of Hyperspectral Data 


As described in Chapter 10, decision trees can be used 
to map the spatial distribution of nominal-scale land 
cover (e.g., urban, wetland, forest) and regression trees 
can be used to extract quantitative information about 
continuous variables (e.g., LAI, biomass, percent cano- 
py cover) (e.g., Im and Jensen, 2008). 
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Mapping LAI and Land Cover of a Capped Waste Site near Monticello, UT, using DecisionTree— 
and Regression Tree—Derived Rules Applied to HyMap Hyperspectral Data acquired on June 2, 2008 
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a. HyMap hyperspectral imagery (RGB = bands 24, 17, 11) 
with 2.3 x 2.3 m spatial resolution overlaid with 
the location of 53 in situ LAI measurements. 


c. Decision tree—derived classiXcation map based on 
analysis of vegetation indices and mixture-tuned- 
matched Xltering (MTMEF) derived metrics. 


b. Predicted LAI distribution based on regression tree— 
rules derived from vegetation indices and mixture- 
tuned matched Mltering (MTMF) derived metrics. 


So Big Sagebrush (Artemisia tridentata) 

@ Rubber Rabbitbrush (Ericameria nauseosa) 
@ Western Wheatgrass (Pascopyrum smithii) 
© Litter (primarily dead grass) 


FIGURE 11-23 a) Color-composite of three of the 126 HyMap hyperspectral bands overlaid with the location of 53 in situ 
sampling locations where LAI and land-cover information were collected. b) LAI spatial distribution based on regression-tree 
machine learning applied to vegetation indices and MTMF-derived metrics. c) Land-cover classification based on decision- 
tree analysis applied to vegetation indices and MTMF-derived metrics (based on Im, J., Jensen, J. R., Jensen, R. R., Gladden, J., 
Waugh, J. and M. Serrato, 2012, “Vegetation Cover Analysis of Hazardous Waste Sites in Utah and Arizona using Hyperspec- 


tral Remote Sensing,” Remote Sensing, 2012(4):327-353). 


Example 1 Im et al. (2012) used decision tree—derived 
rules to map land cover and regression tree—derived 
rules to map the LAI on an engineered waste site in 
Monticello, UT. HyMap airborne hyperspectral data 
were collected in 126 bands from 440 — 2,500 nm at 2.3 
x 2.3 m spatial resolution on June 2-3, 2008 (Figure 
11-23a). The HyMap imagery were atmospherically 
corrected to scaled percent reflectance and polished us- 
ing EFFORT. The polished scaled reflectance data 
were geometrically rectified to a Universal Transverse 
Mercator (UTM) projection using 20 Ground Control 
Points (GCPs) collected from 2006 National Agricul- 
tural Imagery Program (NAIP) digital orthophotos (1 
x 1 m spatial resolution) resulting in a root mean 
square error (RMSE) of <1 pixel. 


LAI and land-cover information were collected in situ 
at the time of the HyMap overflight at the 53 locations 


shown in Figure 11-23a. The dominant species includ- 
ed Big Sagebrush (Artemisia tridentata), Rubber Rab- 
bitbrush (Ericameria nauseosa), Western Wheatgrass 
(Pascopyrum smithii) and dead litter plant material 
(Table 11-4). 


Several information extraction techniques were investi- 
gated in this study but the machine-learning decision- 
tree classifiers yielded in the best results. A machine 
learning regression tree (Cubist) was used to analyze 
hyperspectral vegetation indices and mixture-turned 
matched filtering (MTMF) derived metrics to generate 
a rule-based model of LAI. The usefulness of Cubist 
for creating robust regression trees is well documented 
in the remote sensing literature (e.g., Huang and Town- 
shend, 2003; Moisen et al., 2006; Im et al., 2009). 
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Results of LAI Estimation using 
A Regression Tree Approach 
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FIGURE 11-24 A scatterplot of 53 in situ reference LAI 
measurements versus LAl measurements estimated using a 
regression tree approach applied to HyMap imagery for a 
waste site near Monticello, UT (based on Im et al., 2012, as 
cited in Figure 11-23). 


TABLE 11-4 Field data characteristics for vegetation map- 
ping and leaf-area-index (LAI) estimation on the Monticello, 
UT, waste site in June, 2008 (based on Im et al., 2012, as cit- 
ed in Figure 11-23). 


Number of 
Classes Samples 
Big Sagebrush (Artemisia tridentata) 8 
Rubber Rabbitbrush (Ericameria nauseosa) 12 
Western Wheatgrass (Pascopyrum smithii) 16 
Litter (primarily dead grass) 17 
LAI ranged from 0.09-5.43 53 


LAI estimation using the regression tree approach for 
the Monticello, UT, study area resulted in an R? of 
0.82. A map of the spatial distribution of LAI is shown 
in Figure 11-23b. A scatterplot of the in situ LAI and 
the modeled LAI is shown in Figure 11-24. 


A vegetation species classification map created using 
the decision-tree classifier is shown in Figure 11-23c. 
The decision tree used various vegetation indices plus 
mixture-tuned matched-filtering (MTMF) derived 
variables to generate the rules. The overall classifica- 
tion accuracy was 86.8% with a Kappa Coefficient of 
Agreement of 0.82. The contingency table is provided 
in Im et al. (2012). The study concluded that monitor- 
ing vegetation cover on waste sites using hyperspectral 
remote sensing data and decision-tree machine learn- 
ing is practical although higher spatial resolution im- 
agery (<1 x 1 m) is preferable. 
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Example 2 The previous study focused on the identi- 
fication of LAI and land cover. It is also possible to an- 
alyze hyperspectral remote sensor data to obtain data 
on other biophysical and biochemical measurements. 
For example, Im et al. (2009) used high spatial resolu- 
tion (1 X 1 m) hyperspectral data of experimental short 
rotation woody crop plots on the Savannah River Site 
near Aiken, SC, to extract detailed biophysical and 
biochemical information (Figure 11-25a). 


The objectives of this study were to (1) identify the re- 
lationship between in situ and hyperspectral remote 
sensing measurement in terms of tree species and the 
different levels of water and nutrient availability, and 
(2) estimate biophysical and biochemical characteris- 
tics (i.e. LAI, biomass, leaf nutrients concentrations) 
from the hyperspectral imagery using three different 
digital image processing techniques. 


The leaf nutrient variables (treatments) included nitro- 
gen (N), phosphorous (P), potassium (K), calcium 
(Ca), and magnesium (Mg). The randomized block de- 
sign of the treatments is shown in Figure 11-25b. Treat- 
ments were composed of (1) irrigation only, (2) 
irrigation and fertilization, (3) control (no irrigation 
and fertilization), and (4) fertilization only. Different 
fertilization levels were also applied to experimental 
plots in blocks | and 5, but they were not included in 
this study due to limited in situ measurements. A total 
of 60 plots in blocks 2-4 were used, which included 20 
treatment plots (5 genotypes grown with 2 fertilizer 
and 2 irrigation treatments) with each replicated three 
times (Figure 11-25b). Detailed information about the 
experimental design is found in Coleman et al. (2004). 


An Airborne Imaging Spectrometer for Applications 
(AISA) Eagle sensor system collected hyperspectral 
imagery over the study area on September 15, 2006 
(Figure 11-25a). The imagery consisted of 63 channels 
(from 400 to 980 nm) with a spectral resolution of ap- 
proximately 9-nm, a radiometric resolution of 12-bits, 
and a spatial resolution of | x 1 m. The imagery was 
collected at an altitude of 1,630 m above ground level 
(AGL) during cloud-free conditions at 11:20 a.m. 
(EDI ) local time. 


The hyperspectral imagery were pre-processed to 
scaled percent reflectance using the Fast Line-of-sight 
Atmospheric Analysis of Spectral Hypercube 
(FLAASH) algorithm and rectified to a Universal 
Transverse Mercator (UTM) coordinate system. The 
geometric rectification was performed using numerous 
GPS-derived coordinates located in the study site (i.e., 
each corner of each block) resulting in an estimated 
Root Mean Square Error (RMSE) of 0.48 pixels. 
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Hyperspectral Remote Sensing of Short Rotation Woody Crops Grown 
with Controlled Nutrient and Irrigation Treatments 


Rs. t+ = ~ ~ r - 
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a. AISA hyperspectral 
imagery (RGB = bands 
760.8, 664.4, and 572.5 nm) 
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FIGURE 11-25 a) Color-composite of AISA hyperspectral data at 1X 1 m spatial resolution of short rotation woody crop ex- 
perimental plots. b) Characteristics of the randomized block design associated with vegetation type and fertilization and irri- 
gation treatments. c) Characteristics of the 60 plots used in the study (based on Im, J., Jensen, J. R., Coleman, M., and E. 
Nelson, 2009, “Hyperspectral Remote Sensing Analysis of Short Rotation Woody Crops Grown with Controlled Nutrient and 
Irrigation Treatments,” Geocarto International, 24(4):293-312). 
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TABLE 11-5 Results of predicting biophysical and biochem- 
ical characteristics of short rotation woody crops using AISA 
Eagle hyperspectral imagery (based on Im, J., Jensen, J. R., 
Coleman, M. and E. Nelson, 2009, “Hyperspectral Remote 
Sensing Analysis of Short Rotation Woody Crops Grown with 
Controlled Nutrient and Irrigation. Treatments,” Geocarto 
International, 24(4):293-312). 


Target Variable Calibration R? 

NDVI (using bands centered on 

657.0 and 856.8 nm) 
LAI 0.738 
Biomass 0.569 
N 0.016 
PB 0.019 
K 0.002 
Ca 0.225 
Mg 0.457 

Partial Least Squares Regression 

(using 63 reflectance variables 
LAI 0.854 
Biomass 0.633 
N OMs3) 
P 0.177 
K 0.222 
Ca 0.484 
Mg 0.506 

Machine-learning Regression Tree 

(using 63 reflectance variables 
LAI 0.956 
Biomass 0.855 
N 0.793 
EB 0.229 
K 0.557 
Ca 0.626 
Mg 0.635 


The three digital image processing techniques used 
were (1) simple regression using normalized difference 
vegetation index (NDVI), (2) partial least squares re- 
gression, and (3) machine-learning regression tree. Re- 
sults of the hyperspectral analysis using the three 
techniques are summarized in Table 11-5 (Im et al., 
2009). 


This study demonstrated the capability of hyperspec- 
tral remote sensing to predict selected forest biophysi- 
cal and biochemical characteristics in a very complex 
environment containing five tree genotypes subjected 
to different levels of irrigation and/or fertilization. The 
simple linear regression based on NDVI resulted in 
good performance in predicting the biophysical char- 
acteristics such as LAI and stem biomass for certain 
genotypes. But, it failed to predict the biochemical 
characteristics (i.e., leaf nutrients) covering all tree spe- 
cies and treatments (Table 11-5). The partial least 
squares regression and the regression-tree analysis 
yielded much better estimation of the leaf nutrients 
concentrations (Table 11-5). In particular, the regres- 
sion-tree approach appears to be a robust method for 
predicting biophysical and biochemical properties 
from environmental complex forest environments when 
provided with sufficient high-quality training data. 
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Support Vector Machine (SVM) Analysis of 
Hyperspectral Data 


The SVM classification algorithm described in Chap- 
ter 10 is an effective distribution free classifier that has 
been widely used in the recent years for solving hyper- 
spectral classification problems (Camps-Valls and Bru- 
zzone, 2005; Dalponte et al., 2009). The main reason 
for the choice of this classifier is associated with its 
properties, including: a) high generalization ability and 
high classification accuracies (with respect to other 
classifiers); b) effectiveness in addressing il/l-posed 
problems (which are quite common with hyperspectral 
data mainly because of the limited amount of training 
samples relative to the number of features (bands) 
(Plaza et al., 2009); and c) limited effort required for 
architecture design and training phase when compared 
to other machine-learning algorithms (such as multi- 
layer perceptron neural networks). Dalponte et al. 
(2009) compared SVM, maximum-likelihood, and dis- 
criminant analysis classification techniques for spectral 
resolutions ranging from 4.6 — 36.8 nm and found that 
SVM consistently yielded the highest classification ac- 
curacy, especially when using the highest spectral reso- 
lution hyperspectral data (4.6 nm). Plaza et al. (2009) 
used a suite of SVM to analyze hyperspectral data us- 
ing a variety of techniques that incorporate both spa- 
tial and hyperspectral data. 


Selected Indices Useful for 
Hyperspectral Data Analysis 


Traditional vegetation indices (discussed in Chapter 8) 
developed for use with broadband multispectral data 
can be used with hyperspectral data. In fact, Blonski et 
al. (2002) developed an algorithm that will synthesize 
multispectral bands (e.g., Landsat ETM* bands) from 
hyperspectral data (e.g., AVIRIS), if desired. Several 
studies identify a variety of indices developed especial- 
ly for use with hyperspectral data for a variety of appli- 
cations (e.g., Im et al., 2009; Im et al., 2012; Brantley et 
al., 2011). Thenkabail et al. (2000; 2004; 2011), Habou- 
dane et al. (2002), Stagakis et al. (2010) and Behmann 
et al. (2014) review several narrow-band indices for re- 
mote sensing of agriculture and other types of vegeta- 
tion. Ustin et al. (2009) review indices for retrieving 
plant pigment information using high-resolution spec- 
troscopy. Matthews (2011) provides an overview of in- 
dices used for remote sensing inland and near-coastal 
waters. Alonzo et al. (2014) describe how hyperspectral 
data are used with LiDAR data to perform urban tree 
mapping. Jay et al. (2014) used hyperspectral data to 
map water quality and depth. Selected indices that can 
be applied to hyperspectral data are provided below. 
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Reduced Simple Ratio 


Brown et al. (2000) introduced the Reduced Simple 
Ratio (RSR): 


RSR = Pswir~ Pswirmin | (11.12) 


Phir ( _ 
Pred Pswirmax ~ Pswirmir 


where P gyjrmin ANd P swirmax are the scene minimum 
and maximum short wavelength infrared (swir) reflec- 
tance. Chen et al. (2002) and Gray and Song (2012) 
used the RSR for mapping leaf-area-index. 


Normalized Difference Vegetation 
Index—NDVI 


As discussed in Chapter 8, the normalized difference 
vegetation index is based on the equation: 


Pnir~ Pred 


NDVI = : 
Pir sy Pred 


(11.13) 


Generally, reflectances from a red channel centered 
around 660 nm and a near-infrared channel centered 
on 860 nm are used to calculate the NDVI. The near- 
infrared band empirically corresponds to the long- 
wavelength shoulder of the chlorophyll red-edge, and 
the red band is associated with the maximum chloro- 
phyll absorption. The following is one possible narrow- 
band implementation of the standard NDVI: 


Pg60 ~ Peso 
Pg6o t P660 


NDVI = 


narrow (11.14) 
A narrow-band NDVI image derived from an analysis 
of AVIRIS bands 29 (red) and 51 (near-infrared) is 
shown in Figure 11-26a. As expected, the lower bio- 
mass areas in the NDVI image correspond generally to 
stressed areas in the thematic map. 


Hyperspectral Enhanced Vegetation 
Index—EVI 


Clark et al. (2011) used the following narrow-band hy- 
perspectral Enhanced Vegetation Index: 


P798 ~ P679 


EVI a 
hyper [1 + Page + (6 x PE7q) — (7-5  Pagy)] 


(11.15) 


applied to HYDICE imagery to extract photosynthe- 
sizing vegetation structure, senescence, and health in- 
formation. Behmann et al. (2014) used hyperspectral 
bands in an EVI to assess vegetation stress. 


Yellowness Index—YI 


The yellowness index (YI) constitutes a descriptor of 
leaf chlorosis exhibited by stressed vegetation. It mea- 
sures the change in shape of the reflectance spectra in 
the interval between the 0.55-Xm reflectance maximum 
and the 0.65-%m minimum. The YI is a three-point 
gross measure of green—red spectral shape and is thus 
computed using only wavelengths in the visible spec- 
trum. The justification is that the specified visible re- 
gion tends to be relatively insensitive to changing leaf 
water content and structure (Philpot, 1991; Adams et 
al., 1999): 


= 2 
vie POD =2°C0) PCa) dp 


AX? dh 


(11.16) 


where p(A,) is the band center reflectance; p(A,) and 
p(A,,) denote the reflectance of the lower and higher 
wavebands, respectively; and AA is the spectral dis- 
tance (measured in Km) between wavebands (i.e., 
Ahk = hg -A_1 = A41 Ag). The goal is to select wave- 
lengths such that the band separation (Ad ) is as large 
as possible while constraining all three channels in the 
spectral range between approximately 0.55 Xm and 
0.68 Xm. The YI can be computed as 0.1 multiplied by 
the negative of the finite approximation of the second 
derivative as a means of downscaling the range of out- 
put values; thus, the resultant relationship would be 
that as yellowness increases, so do positive YI values 
(Adams et al., 1999). The YI is in units of relative re- 
flectance m’* (RRU &m). The YI magnitude is sensi- 
tive to the 4, and Ad values. Using 1999 AVIRIS 
data, each value in the YI equation can be the average 
of three adjacent bands; 4, centered at 0.61608 Xm 
(band 26); A_, centered at 0.56696 Mim (band 21); and 
A,, centered at 0.66518 Mm (band 31) based on Ad- 
ams et al. (1999). 


Physiological Reflectance Index—PRI 


The physiological reflectance index (PRI) is a narrow- 
band index that has been correlated with the epoxida- 
tion state of the xanthophyll pigments and with photo- 
synthetic efficiency with respect to control (unstressed) 
and nitrogen-stressed canopies (Gamon et al., 1992). 
However, the PRI is generally not well correlated with 
water-stressed canopies experiencing midday wilting. 
The PRI employs the reflectance at 531 nm and a refer- 
ence channel in order to minimize the effects of diurnal 
Sun angle changes. The PRI can track diurnal changes 
in photosynthetic efficiency. In addition, the PRI may 
be useful in situations where in situ spectroradiometer 
data are acquired at a different time or light regime ge- 
ometry than the remote sensor data. The general PRI 
formula is: 
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Normalized Difference Vegetation Index (NDVI) 


Image Derived from AVIRIS Data 
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a. NDVI image derived from AVIRIS bands 
29 (0.64554 Xm) and 51 (0.82593 um). 
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Second-order Derivative Image 
Extracted from AVIRIS Data 


b. Second-order derivative image. 


FIGURE 11-26 a) Normalized difference vegetation index (NDVI) image derived from AVIRIS bands 29 (0.64554 Xm) and 51 
(0.82593 Xm), red and near-infrared, respectively. White and red pixels indicate the greatest amount of biomass. Blue, green, 
and orange pixels contain less biomass. b) A second-order derivative image extracted from the original AVIRIS scaled surface 
reflectance data (RGB = bands 42, 30, 18). Original AVIRIS imagery courtesy of NASA. 


pri = Pref Ps31 (11.17) 
Prep t P531 

where p,.,¢ is a reference wavelength and p.3, is the 

reflectance at 531 nm. The best PRI noted in Gamon 

et al. (1992) was: 


P5507 P531 


PRI = . 
P5590 + P531 


(11.18) 


A reference wavelength of 550 nm seems to be appro- 
priate at the canopy level. For leaf-scale spectra, a ref- 
erence wavelength of 570 nm is likely better for a 
xanthophyll signal. Note that a single PRI is not likely 
to be applicable across all spatial and temporal scales 
as well as various canopy types and diurnally dynamic 
canopy structures. AVIRIS implementation for the 
1999 data might include bands 19 (centered at 547.32 
nm) and 17 (centered at 527.67 nm), respectively. 


In contrast with the PRI, the NDVI does not accurate- 
ly indicate real-time photosynthetic fluxes. While 
NDVI is often sensitive at low leaf-area-index (LAI) 
values, it often saturates at high LAI values. The PRI 
may be able to indicate short-term changes in photo- 
synthetic efficiency, especially in canopies with high 
LAI values where the NDVI is least effective (Gamon 
et al., 1992). 


Normalized Difference Water Index 
—NDWI 


The normalized difference water index (NDWI) may 
be used to remotely determine vegetation liquid water 


content. Two near-infrared channels are used in the 
computation of the NDWI; one is centered at approxi- 
mately 860 nm and the other at 1,240 nm (Gao, 1996; 
Gray and Song, 2012): 


Pg60~ P1240 


NDWI = 
Pg60 + P1240 


(11.19) 


AVIRIS bands that might be used include bands 55 
(864.12 nm) and 94 (1237.94 nm). Clark et al. (2011) 
used HYDICE wavelengths of 862 and 1,239 nm. 


Linear Red-edge Position—REP 


The red-edge position (REP) is defined as the point of 
maximum slope on a vegetation reflectance spectrum 
between the red and near-IR wavelengths. The REP is 
useful because it is strongly correlated with foliar chlo- 
rophyll content and can be a sensitive indicator of veg- 
etation stress. Although the AVIRIS sensor has a 
nominal band separation of 10 nm, subtle REP shifts 
may not be discernible because the spectra are still 
sampled coarsely relative to data from laboratory spec- 
trometers (Dawson and Curran, 1998). 


A linear method proposed by Clevers (1994) can be im- 
plemented that makes use of four narrow bands and is 
computed as (Im et al., 2012): 


p -p 
REP = 700 + so [Peer (11.20) 


P740~ P700 


where 
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_ Pozo * P780 


Pred edge 2 (11.21) 


A derivative-based REP algorithm exists that is of low 
computational complexity and is appropriate for some 
canopy-scale studies (Dawson and Curran, 1998). 


Red-Edge Vegetation Stress Index 
(RVSI) 


Clark et al. (2011) used the following narrow-band hy- 
perspectral Red-edge Vegetation Stress Index (RVSI) 


Pag.” v2) : 
~ F730 


RVSI = ( : 


(11.22) 


applied to HYDICE imagery to extract photosynthe- 
sizing vegetation stress information. 


Crop Chlorophyll Content Prediction 


Haboudane et al. (2002) developed a narrow-band veg- 
etation index that integrates the advantages of indices 
that minimize soil background effects and indices that 
are sensitive to chlorophyll concentration. The Trans- 
formed Chlorophyll Absorption in Reflectance Index 
(TCARDJ) (Daughtry et al., 2000) is: 


p 
TCARI = 3 (n99~ P6ro)~ 920799 550 = (11.23) 
The Optimized Soil-Adjusted Vegetation Index (Ron- 


deaux et al., 1996) belongs to the Soil-Adjusted Vege- 
tation Index (SAVI) family (Huete, 1988) and is: 


_ (1+ 0.16)(Pgo9 — P6709) 


OSAVI = (11.24) 
(Pgog + P670 + 0.16) 
The ratio 
TCARI (11.25) 
OSAVI 


is sensitive to chlorophyll content variations and resis- 
tant to variations of leaf-area-index (LAI) and solar 
zenith angle. Evaluation with ground truth resulted in 
an r? = 0.81 between estimated and field-measured 
chlorophyll content data. Haboudane et al. (2002) sug- 
gest that the ratio index is insensitive to LAI variations 
for LAI values ranging from 0.5 to 8 and that it might 
be used operationally in precision agriculture because 
it allows an accurate estimation of crop photosynthetic 
pigments without a priori knowledge of crop canopy 
architecture. 


Modified Chlorophyll Absorption 
Ratio Index (MCARI1) 


Haboudane et al. (2002) proposed the modified chloro- 
phyll absorption ratio index (MCARI]1): 


MCARIL = 1.2[2.5(pgqq + P6790) - 1:3 (Pgqq - P550)1 (11.26) 
which is more sensitive to variations in green LAI. 


Chlorophyll Index 


Brantley et al. (2011) used a chlorophyll index (CI) 
based on the work of Gitelson et al. (2005) that makes 
use of red-edge wavelengths: 


Cl = P750 


= — 70 (11.27) 
(P7099 + P7109) — 1 


It is a good predictor of chlorophyll content for vegeta- 
tion with high LAI. 


Medium Resolution Imaging 
Spectrometer (MERIS) Terrestrial 
Chlorophyll Index (MTCI) 

Dash et al. (2010) used a European Space Agency Env- 
isat product, the Medium Resolution Imaging Spec- 
trometer (MERIS) Terrestrial Chlorophyll Index 


(MTC), to study the spatio-temporal variation of veg- 
etation phenology over India: 


MERIS) and10 = MERIS,, ond9 
MERIS,, ond9 ~ MERIS,, ond 


MTCI = (11.28) 


mtTcy = £253.75 P708.75- 
P708.75 — 9681.25 


This index is highly correlated with chlorophyll con- 
centration and leaf-area-index and has limited sensitiv- 
ity to atmospheric effects, soil background, and view 
angle. 


He Derivative Spectroscopy 


Differentiation of a curve or its mathematical function 
estimates the slope over the entire interval. Deriving 
the slope of spectroradiometer-derived curves is called 
derivative spectroscopy. Derivative spectroscopy meth- 
ods were originally developed in analytical chemistry 
to eliminate background signals and resolve overlap- 
ping spectral features (Demetriades-Shah et al., 1990). 
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The concept has also been applied to differentiating re- 
mote sensing—derived spectra. Differentiation does not 
yield more information than exists in the original spec- 
tral channels. It can be used, however, to emphasize de- 
sired information while suppressing or eliminating 
information not of interest. For example, background 
absorption or reflectance signals caused by stray light 
can be eliminated (Talsky, 1994). Spectral features with 
sharp structures may be enhanced compared with 
broader-structured features. 


Because of the inherent differences between lab- and 
remote sensor—based data, not all lab-based spectro- 
scopic procedures translate well to remotely sensed in- 
quiries. Lab analysis is characterized by controlled 
illumination sources and viewing geometries, as well as 
by the typical assumption of homogenous target sam- 
ples and the use of a known standard. Conversely, im- 
aging spectroscopic remote sensing entails a natural 
illumination source, mixed pixels, varying topography, 
generally coarser spectral resolution than that of labo- 
ratory spectrophotometers, and the general lack of 
useful reference standards (Tsai and Philpot, 1998). 
Given these differences, the application of laboratory 
spectroscopic techniques to remote sensor data must 
be carefully considered. Derivative techniques have 
been applied to remote sensor imagery, but such appli- 
cations have been rather limited to date (e.g., 2 metri- 
ades-Shah et al., 1990; Philpot, 1991; Li et al., 1993; 
Penuelas et al., 1993; Tsai and Philpot, 1998; Thenka- 
bail et al., 2004; Mitchell et al., 2012). Nevertheless, 
there are advantages to using derivative-based analyti- 
cal techniques with remotely sensed data. Derivative 
spectra are sometimes more informative than zero-or- 
der reflectance spectra when attempting to glean cer- 
tain information or relationships from the data. For 
example, Malthus and Madeira (1993) found the first- 
order derivative spectra in the visible wavelengths to be 
more highly correlated with percent leaf surface area 
infected by the fungus Botrytis fabae than the original 
zero-order reflectance data. 


First-, second-, and third-order derivative spectra may 
be computed on a pixel-by-pixel basis. The differentia- 
tion can be performed using three-point Lagrangian 
formulas (Hildebrand, 1956): 


2 
= h 
a4 = Ass 3a_, + 4ay-a,) + 5 a5) (11.29) 
I he 
ag = Ae FI wah) els) (11.30) 


oy h 


INFORMATION EXTRACTION USING IMAGING SPECTROSCOPY 495 


where the subscripts 0, -1, and | denote the first deriv- 
ative at the center point, and points to the left and 
right of the center point, respectively. h and € are dis- 
tance and error terms, respectively. Higher-order deriv- 
ative spectra are calculated in an analogous manner. 


In remote sensing investigations, higher-order deriva- 
tives (second-order and higher) are relatively insensi- 
tive to illumination intensity variations, due to cloud 
cover, Sun angle variance, or topographic effects. In 
addition, derivatives are usually insensitive to changes 
in spectral solar flux and skylight (Tsai and Philpot, 
1998). Derivative techniques can also be used to ad- 
dress interference from soil background reflectance in 
vegetation remote sensing studies (i.e., separating the 
vegetative signal from the background noise). Second- 
order derivative spectra, which are insensitive to soil 
reflectance, specifically mitigate this problem, while 
first-order derivative spectra do not (Demetriades- 
Shah et al., 1990; Li et al., 1993). 


Narrow-Band Derivative-Based 
Vegetation Indices 


A variety of narrow-band derivative-based vegetation 
indices may be computed. For instance, several deriva- 
tive vegetation indices that measure the amplitude of 
the chlorophyll red-edge using hyperspectral data in 
the 626-nm to 795-nm spectral range can be imple- 
mented (Elvidge and Chen, 1995): 


An 
IDL_DGVI = S"[p'(A,)—p'(A)JAa; (11.32) 
hy 
An 
IDZ_DGVI = S[p'(,)|44, (11.33) 
dy 
hn 
2DZ_DGVI = *[p"(2,)|A4,; (11.34) 
hy 
An 
3DZ_DGVI = J "|p"(2,)|A4,; (11.35) 
dy 


where 7 is the band number; rj is the center wavelength 
for the ith band; A, = 626 nm; A, = 795 nm; p’ is the 
first derivative reflectance; and p’’ is the second deriv- 
ative reflectance. Higher-order derivative-based indices 
are also possible. For instance, p’’’ is the third-order 
derivative reflectance. Thenkabail et al. (2004) used 
this index to determine the best hyperspectral wave- 
bands for studying vegetation and agricultural crops. 
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The 16-bit radiometric resolution of AVIRIS data fa- 
cilitates the creation of quality higher-order derivative 
spectra. For example, a second-order derivative image 
of the MWMEF on the Savannah River Site is displayed 
in Figure 11-26b. Approximately the same areas of 
healthy vegetation and potential stressed vegetation 
are apparent in the derivative image, but they are more 
visually striking. 


Mitchell et al. (2012) analyzed HyMap hyperspectal 
data using derivative analysis to map semiarid shru- 
bland Nitrogen (N) content. They were able to quanti- 
fy sagebrush canopy N concentrations using partial 
least squares regression with an R? of 0.72. 


Red-Edge Position Based on 
Derivative Ratio 
Smith et al. (2004) used a red-edge positioning ap- 


proach where reflectance data were first smoothed us- 
ing a weighted mean moving average function: 


(>726 + P7094 J 7 
smooth smoot. 
Oh. ae a. 7 (11.36) 
red-edge (6 +p Ja 
103 mooth 701 mooth 
where 
p = 


die smooth 


0.25-p +0.5-p 
ee Aged 
0.25+0.5+1+0.5 + 0.25 


+p, +0.5-p +0.25-p 
My wea ee 


The function used a 5-nm sample range, which consists 
of five values along the reflectance continuum. The rel- 
ative weights of 0.25, 0.5, 1, 0.5, and 0.25 are applied to 
the values of the reflectance continuum to compute the 
average value. The derivative is then calculated by di- 
viding the difference between two average reflectance 
values with a 2-nm interval. Additional methods used 
to determine the red-edge position are summarized in 
Baranoski and Rokne (2005). 
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Biophysical materials and human-made features on the 
surface of Earth are inventoried using remote sensing 
and in situ techniques. Some of the data are fairly stat- 
ic; they change very little over time. Conversely, some 
biophysical materials and human-made features are 
dynamic, changing rapidly. It is important that such 
changes be inventoried accurately so that the physical 
and human processes at work can be more fully under- 
stood (e.g., Miller et al., 2003; Leprince et al., 2007; 
Jensen et al., 2012; Kit and Ludeke, 2013). It is not sur- 
prising, therefore, that significant effort has gone into 
the development of remote sensing change detection 
methods (e.g., Patra et al., 2013; Zhu and Woodcock, 
2014; Rokni et al., 2015). 


Land-use/land-cover change is a major variable im- 
pacting global climate change (Foley et al., 2005). This 
is one reason that the Multi-Resolution Land Charac- 
teristics (MRLC) partners who guide the creation of 
the U.S. National Land Cover Database (NLCD) 
products have shifted the emphasis from characterizing 
land cover every ten years to monitoring land cover 
change every five years (Fry et al., 2011). 


a Overview 


This chapter reviews how change information is ex- 
tracted from remotely sensed data using digital image 
processing techniques. It summarizes the remote sen- 
sor system and environmental parameters that must be 
considered when conducting change detection. Several 
of the most widely used change detection algorithms 
are introduced and demonstrated. 


CHANGE DETECTION 


Change in NDVI 1990 — 2000 
~ a : t 


Source: USGS and NASA 


» Steps Required to Perform 
Be) Change Detection 


The general steps required to perform digital change 
detection using remotely sensed data are summarized 
in Figure 12-1. It is instructive to review the character- 
istics of these steps. 


Specify the Thematic Attribute(s) or 
Indicator(s) of Interest 


Clear identification of the attributes or indicators of in- 
terest is required. Perhaps you are interested in docu- 
menting the change in vegetation species distribution 
through time or monitoring urban expansion into ru- 
ral areas. Perhaps you want to immediately monitor 
the location of areas inundated by floodwaters or the 
location of affected trees as a disease migrates through 
a forest. Common human-made and natural resource 
attributes or indicators that are often the focus of land- 
use planning and natural resource monitoring are sum- 
marized in Table 12-1 (Kennedy et al., 2009). 


Specify the Change Detection 
Geographic Region of Interest (ROI) 


In conjunction with the specification of the thematic 
attribute or indicator of interest, the dimensions of the 
change detection region of interest (ROI) must be care- 
fully identified and held constant throughout a change 
detection project. The geographic ROI (e.g., a county, 
state, region or watershed) is especially important in a 
change detection study because it must be completely 
covered by n dates of imagery. Failure to ensure that 
each of the multiple-date images covers the geographic 
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TABLE 12-1 Common human-made and natural resource attributes or indicators that are often the focus of land-use planning 
and natural resource monitoring programs (expanded from Kennedy et al., 2009). 


Man-made and/or Natural Resource 
Attributes or Indicators 


Process of Interest or Threat 


Change in the size or shape of areal patches 
(polygons) of related cover types 


Urban expansion (sprawl); vegetation expansion; sea-level rise; consolida- 
tion, fragmentation; infilling; encroachment; erosion/dilution 


Change in width or character of linear 
features 


Densification of road, utility, or hydrologic network; impact of use of paths 
or roads; impact of flooding on riparian vegetation; dynamics of terrestrial 
and submerged near-shore aquatic vegetation 


Slow changes in surface cover types or 
species composition 


Succession, competition, eutrophication, consolidation, fragmentation, 
exotic species invasion 


Abrupt changes in surface cover, water, and/ 
or atmospheric condition 


Catastrophic event (e.g., hurricane, flood, tornado, volcanic eruption, fire, 
wind, landslides), disturbance, human activity (e.g., land clearing; urban 
and/or environmental terrorism), land management practices (e.g., no-till 
farming; prescribed burning) 


Slow changes in condition of a single cover 


Climate-related changes in vegetation species composition and/or pro- 
type ductivity; sea surface temperature; slowly-spreading forest mortality 
caused by insect or diseases; changes in moisture regime 


Changes in timing or the extent of diurnal 
and/or seasonal processes 


Coastal zone dynamics, snow cover dynamics, natural vegetation and agri- 
culture phenology 


area of interest results in change detection maps with 
data voids that are problematic when computing 
change statistics. 


Specify the Change Detection Time 
Period 


Sometimes change detection studies are overly ambi- 
tious in their attempt to monitor changes in the land- 
scape. Sometimes the time period selected over which 
change is to be monitored is too short or too long to 
capture the information of interest. Therefore, the ana- 
lyst must be careful to identify the optimal change de- 
tection time period(s). This selection, of course, is 
dictated by the nature of the problem. Traffic transpor- 
tation studies might require a change detection period 
of just a few seconds or minutes. Conversely, images 
obtained monthly or seasonally might be sufficient to 
monitor the greening-up of a continent. Careful selec- 
tion of the change detection time period can ensure 
that resource analysis funds are not wasted. 


It is important to note that just because multiple-date 
remote sensing imagery is available does not ensure 
that change detection will be successful. The multiple- 
date imagery may have been acquired for an entirely 
different purpose. The most effective change detection 
projects are those based on timely multiple-date imag- 
ery selected specifically for the project that meet the 
system, environmental, and bureaucratic requirements 
to be discussed. 


Select an Appropriate Land-Use/ 
Land-Cover Classification System 
As discussed in Chapter 9, it is wise to use an estab- 


lished, standardized land-use/land-cover classification 
system for change detection, such as the following: 


¢« American Planning Association Land-Based Classi- 
fication Standard (LBCS); 

*« U.S. Geological Survey Land UselLand Cover Clas- 

sification System for Use with Remote Sensor Data; 

¢« US. National Land Cover Dataset (NLCD) Classi- 

fication scheme; 

¢ NOAA Coastal Change Analysis Program (C-CAP) 

classification scheme; 


¢ U.S. Department of the Interior Fish & Wildlife Ser- 
vice, Classification of Wetlands and Deepwater Habi- 
tats of the United States; 


¢ US. National Vegetation & Classification Standard; 


¢ International Geosphere-Biosphere Program Land 
Cover Classification System. 


The use of a standardized classification system allows 
change information derived from one project to be 
compared with change derived from another project. 


Select Hard (Crisp) and/or Soft 
(Fuzzy) Change Detection Logic 
Most change detection studies have been based on the 


comparison of multiple-date hard (crisp) land-cover 
classifications of remotely sensed data. The result is the 
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General Steps Used to Conduct Digital 
Change Detection Using Remote Sensor Data 


State the nature of the change detection problem. 
* Specify thematic attribute(s) or indicator(s) of interest. 
* Specify change detection geographic region of interest (ROI). 
* Specify change detection time period (e.g., daily, seasonal, yearly). 
* DeXne the classes of interest in a classiXcation system. 
* Select hard and/or fuzzy change detection logic. 
* Select per-pixel or object-based (OBIA) change detection. 
Considerations of significance when performing change detection. 
* Remote sensing system considerations: 
- Spatial, spectral, temporal, and radiometric resolution 
* Environmental considerations: 
- Atmospheric conditions 
- Cloud cover and cloud shadow 
- Relative humidity 
- Soil moisture conditions 
- Phenological cycle characteristics 
- natural (e.g., vegetation, soil, water, snow and ice) 
- man-made phenomena 
- Obscuration considerations 
- natural (e.g., trees, shadows) 
- man-made (e.g., structures, shadows) 
- Tidal stage, etc. 
Process remote sensor data to extract change information. 
* Acquire appropriate change detection data: 
- In situ ground reference information 
- Collateral data (e.g., soil maps, parcels, digital elevation model) 
- Remotely sensed data: 
- Base year (time 7) 
- Subsequent year(s) (time n— 1 orn + 1) 
* Preprocess the multiple-date remote sensor data: 
- Geometric correction 
- Radiometric correction (or normalization) if necessary 
* Select change detection algorithm. 
- Binary “change versus non-change” algorithm 
- Thematic “from-to” algorithm 
* Apply appropriate image classiXcation logic if necessary: 
- Method (supervised, unsupervised, hybrid) 
- Parametric (e.g., maximum-likelihood) 
- Non-parametric (e.g., decision or regression trees, random forests) 
* Perform change detection using GIS algorithms if necessary: 
- Highlight selected classes using change detection matrix 
- Generate change-map products 
- Compute change statistics 
Perform accuracy assessment. 
* Select method: 
- Q alitative conXdence building 
- Statistical measurement 
* Determine number of samples required by class. 
* Select sampling scheme. 
* Obtain ground reference test information. 
* Create and analyze change detection error matrix: 
- Univariate and multivariate statistical analysis 
Accept or reject previously stated hypothesis. 
Distribute results if accuracy is acceptable. 


FIGURE 12-1 The general steps used to perform digital change detection of remotely sensed data. 
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creation of a hard change detection map consisting of 
information about the change in discrete categories 
(e.g., change in forest, agriculture). This is still very im- 
portant and practical in many instances, but we now 
recognize that it is sometimes useful to capture both 
discrete and soft (fuzzy) changes in the landscape (refer 
to Chapter 9 for a discussion about fuzzy land-cover 
classification). 


Land-cover changes may range from no landscape al- 
teration whatsoever, through modifications of variable 
intensity, to a full transformation or conversion to an 
entirely new class (e.g., see the Denver, CO, example 
later in this chapter). Scientists now believe that re- 
placing the Date n and Date n + J hard classification 
maps typically used in a change detection project with 
fuzzy classification maps can result in more informa- 
tive and accurate land-cover change information 
(Foody, 2001; NRC, 2010). However, as discussed in 
Chapter 13, it is more difficult to determine the accu- 
racy of soft (fuzzy) change detection map products. 


Select Per-pixel or Object-Based 
Change Detection (OBCD) 


The majority of digital image change detection has been 
based on processing Date n and Date n + / classifica- 
tion maps pixel by pixel. This is commonly referred to 
as per pixel change detection. Conversely, object-based 
change detection (OBCD) involves the comparison of 
two or more scenes consisting of many homogenous 
image objects (i.e., patches) that were identified using 
the object-based image analysis (OBIA) techniques 
discussed in Chapter 9 (e.g., Im, 2006; Hofmann et al., 
2008; Tsai et al., 2011; Gartner et al., 2014). The ho- 
mogeneous image objects (i.e., polygons) in the two 
scenes are then subjected to change detection tech- 
niques discussed in this chapter. 


Remote Sensing System Change 
Detection Considerations 


Successful remote sensing change detection requires 
careful attention to: 


* remote sensor system considerations, and 
* environmental characteristics. 


Failure to understand the impact of the various param- 
eters on the change detection process can lead to inac- 
curate results (Lunetta and Elvidge, 2000; Jensen et al., 
2009). Ideally, the remotely sensed data used to per- 
form change detection is acquired by a remote sensor 
system that holds the following resolutions constant: 
temporal resolution, look angle, and spatial, spectral, 
and radiometric resolution. It is instructive to review 
each of these parameters and identify why they can 


have a significant impact on the success of a remote 
sensing change detection project. Kennedy et al. (2009) 
provide additional guidance on the spatial, spectral, 
and temporal characteristics of the remote sensor data 
for a variety of change detection applications. 


Temporal Resolution 


Two important temporal resolutions should be held 
constant during change detection using multiple dates 
of remotely sensed data, if possible. Ideally, the remote 
sensing data used in the change detection should be 
obtained from a sensor system that acquires data at ap- 
proximately the same time of day. For example, Land- 
sat Thematic Mapper (TM) data are acquired before 
9:45 a.m. for most of the conterminous United States. 
This eliminates diurnal Sun angle effects that can cause 
anomalous differences in the reflectance properties of 
the remotely sensed data. 


Whenever possible it is desirable to use remotely sensed 
data acquired on anniversary dates; for example, June 
1, 2012, and June 1, 2013. Using anniversary date im- 
agery minimizes the influence of seasonal Sun-angle 
and plant phenological differences that can negatively 
impact a change detection project (Dobson et al., 1995; 
Jensen, 2007; Chen et al., 2012). In addition, all of the 
shadows found in the anniversary date imagery ob- 
tained by the same sensor system at approximately the 
same time of day have the same shadow orientation 
and length. This is very important when extracting 
change information from high spatial resolution imag- 
ery because changes in shadow length and orientation 
can introduce many inaccurate change artifacts into 
the study (Jensen et al., 2012). 


Of course, anniversary date imagery is not practical 
when monitoring dynamic change associated with for- 
est fires, floods, hurricanes, volcanic eruptions and 
when monitoring seasonal vegetation (e.g., agriculture) 
(Chen et al., 2012). It is also important to note that 
while many change detection projects deal with just 
two images at a time, there are also studies that utilize 
dense ftime-stacks of multiple-date imagery in the 
change detection process (e.g., Lunetta et al., 2006; 
Schneider, 2012). 


It is useful to consider a hypothetical change detection 
project to demonstrate several variable considerations. 
For example, suppose it was necessary to map the 
change in land cover of San Francisco, CA, over a 
specified time period using high spatial resolution Dig- 
italGlobe QuickBird imagery consisting of both pan- 
chromatic and multispectral data collected in 2005, 
2007, 2009, and 2011. For bureaucratic reasons it is 
necessary to only use imagery collected from October 
Ist through December 15th (bureaucratic constraints 
are often one of the most difficult criteria to satisfy in a 


TABLE 12-2 DigitalGlobe, Inc., QuickBird imagery available 
for change detection in 2005, 2007, 2009, and 2011 with the 
follow ng criteria: imager y collected from Oct. 1-Dec. 15; 
maximum cloud cover (20%); maximum off-Nadir look angle 
(30°); minimum Sun elevation angle (25°) (information cour- 
tesy of DigitalGlobe, Inc.). 


Maximum Minimum 
Acquisition Off-Nadir Sun Cloud 
Date look-angle | Elevation Cover 
year-mo-day (°) (°) (%) 
2005.10.05 24.45 46.58 0 
2005.11.05 er35) 35.88 0 
2005.11.23 16.72 31.25 0 
2007.11.11 26.63 34.28 0 
2009.10.16 3.11 41.71 0 
2009.11.21 9.48 30.95 0 
2011.11.04 24.58 34.03 0 
2011.12.03 27.40 27.25 0 


change detection project). We would want imagery that 
has <20% cloud cover, a minimum Sun elevation of 
25°, and off-Nadir look angles of <30° (to be dis- 
cussed). Actual images in the DigitalGlobe, Inc., ar- 
chive that meet these selected criteria are listed in Table 
12-2. Note that because it is late in the year (October— 
December) in the northern hemisphere that the mini- 
mum Sun angles are low as expected but all exceed 25°. 
Based on these criteria it appears that there are suffi- 
cient dates in the archive to conduct the change detec- 
tion project as all of the images have 0% cloud cover. 
The analyst would have to select a single image from 
each date to complete the project after considering 
look angle. 


Look Angle 


Some remote sensing systems like GeoEye-1, Quick- 
Bird, World View-2, and airborne sensors (e.g., Pictom- 
etry) collect data at off-Nadir /Jook angles as much as 
+30K that is, the sensors obtain data of an area on the 
ground from an oblique vantage point. Imagery used in 
the change detection process with significantly differ- 
ent look angles can cause problems. For example, con- 
sider a maple forest consisting of very large, randomly 
spaced trees. A QuickBird image acquired at approxi- 
mately OM off-Nadir will look directly down on the top 
of the canopy. Conversely, a QuickBird image acquired 
at 27M off-Nadir will record reflectance information 
from the side of the canopy. Differences in reflectance 
from the two datasets may cause spurious change de- 
tection results. Therefore, the data used in a remote 
sensing digital change detection should be acquired 
with approximately the same look angle, if possible 
(Chen et al., 2012). Most change detection algorithms 
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do not work well when the look angle on one date of 
imagery is dramatically different from the look angle 
on the other date(s) of imagery. Of course, imagery ac- 
quired at nadir is preferable. 


A review of the imagery available for the hypothetical 
San Francisco, CA, change detection study summa- 
rized in Table 12-2 reveals that there are significant dif- 
ferences between the maximum off-Nadir look angles. 
Three of the years (2005, 2007, and 2011) have look 
angles of >13°. Only the 2009 imagery has look angles 
<10°. The analyst conducting the change detection 
study would have to carefully weight the importance of 
look angle in conjunction with the other variables 
when selecting the imagery to be analyzed. 


Spatial Resolution 


Accurate spatial registration of at least two images is 
essential for digital change detection (Roy, 2000; Ken- 
nedy et al., 2009; Klemas, 2011). Ideally, the remotely 
sensed data are acquired by a sensor system that col- 
lects data with the same instantaneous field-of-view (IF- 
OV) on each date. For example, Landsat Thematic 
Mapper data collected at 30K 30 m spatial resolution 
on two dates are relatively easy to register to one an- 
other. However, it is possible to perform change detec- 
tion using data collected from two different sensor 
systems with different IFOVs, for example, Landsat 
TM data (30 30 m) for Date 1 and SPOT HRV XS 
data (20 X 20 m) for Date 2. In such cases, it is usually 
necessary to decide on a representative minimum map- 
ping unit (e.g., 20M 20 m) and then resample both da- 
tasets to this uniform pixel size. This does not present a 
significant problem as long as the image analyst re- 
members that the information content of the resam- 
pled data can never be greater than the IFOV of the 
original sensor system (i.e., even though the Landsat 
TM data may be resampled to 20 X 20 m pixels, the in- 
formation was still acquired at 30K 30 m resolution 
and we should not expect to be able to extract addi- 
tional spatial detail from the TM dataset). 


Geometric rectification algorithms (Chapter 6) are 
used to register the images to a standard map projec- 
tion or coordinate system (e.g., Universal Transverse 
Mercator). Rectification should result in the two imag- 
es having a root mean square error (RMSE) of < 0.5 
pixel. Misregistration of the two images may result in 
the identification of spurious areas of change between 
the datasets (Klemas, 2011). For example, just one pix- 
el misregistration may cause a stable road on the two 
dates to show up as a new road in the change image. 


Spectral Resolution 


A fundamental assumption of digital change detection 
is that a difference exists in the spectral response of a 
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pixel on two dates if the biophysical materials within 
the IFOV have changed between dates. Ideally, the 
spectral resolution of the remote sensor system is suffi- 
cient to record reflected radiant flux in spectral regions 
that best capture the most descriptive spectral attri- 
butes of the object. However, different sensor systems 
do not record energy in exactly the same regions of the 
electromagnetic spectrum (i.e., bandwidths). For exam- 
ple, the SPOT 1, 2, and 3 HRV sensors collected data 
in three relatively broad multispectral bands and one 
panchromatic band. The Landsat ETM” collects data 
in six relatively broad optical bands, one thermal infra- 
red band, and one broad panchromatic band (Chapter 
2). Ideally, the same sensor system is used to acquire 
imagery on multiple dates. When this is not possible, 
the analyst should select bands that approximate one 
another. For example, SPOT bands | (green), 2 (red), 
and 3 (near-infrared), can be used successfully with 
Landsat ETM™ bands 2 (green), 3 (red), and 4 (near- 
infrared). Many of the change detection algorithms do 
not function well when the spectral characteristics of 
bands from one sensor system do not match those of 
another sensor system (Jensen et al., 2012). 


Radiometric Resolution 


As previously mentioned in Chapter 2, it is possible to 
digitize analog aerial photography and convert it into 
digital remote sensor data at a specific spatial resolu- 
tion (e.g., | X 1 m) and a specific radiometric resolu- 
tion (e.g., 8-bit values from 0 to 255) and then use the 
data for change detection purposes. Conversely, most 
modern digital remote sensing systems routinely col- 
lect data at radiometric resolutions ranging from 8-bit 
(values from 0 to 255) to 11-bit (values from 0 to 
2,047). For example, GeoEye-1 and WorldView-2 re- 
mote sensing systems collect multispectral data with 
11-bit radiometric resolution. Generally, the higher the 
radiometric resolution, the greater the probability that 
just noticeable differences in spectral reflectance will be 
able to be discerned between various objects on the ter- 
rain (e.g., between building rooftop materials and sur- 
rounding tree and grass vegetation). 


Ideally, remote sensor data are collected on multiple 
dates using the identical remote sensing system, which 
holds the radiometric resolution variable constant. 
However, if one of the dates of imagery has a radiomet- 
ric resolution that is lower than the other (e.g., one 
consists of 8-bit data and one consists of 10-bit data), 
then it is general practice to transform the lower reso- 
lution data (e.g., 8-bit with values from 0 to 255) to the 
same radiometric level as the superior radiometric res- 
olution (e.g., 10-bit with values from 0 to 1,023). Of 
course, the radiometric characteristics of the 8-bit data 
are not actually improved during this process. But the 
transformed 8-bit data now have a range of values that 
is compatible with the 10-bit data. 


An even more elegant solution is to convert the 8-bit 
data to scaled percent reflectance (values from 0 to 
100%) and convert the 10-bit data to scaled percent re- 
flectance (values from 0 to 100%). This is common 
practice when monitoring continuous biophysical vari- 
ables such as leaf-area-index (LAI) or biomass through 
time. However, the use of scaled percent reflectance 
data is not always mandatory when conducting urban 
image classification and change detection. Similarly, 
most of the multiple date digital imagery analyzed us- 
ing soft-copy photogrammetric techniques is not trans- 
formed into scaled percent reflectance values prior to 
processing (Jensen et al., 2012). 


Environmental/Developmental 
Considerations of Importance When 
Performing Change Detection 


Failure to understand the impact of various environ- 
mental characteristics on the remote sensing change 
detection process can lead to inaccurate results. When 
performing change detection, it is desirable to hold as 
many environmental variables as constant as possible 
(Jensen et al., 2009). 


Atmospheric Conditions 


Ideally, the sky has 0% cloud cover and there is low rel- 
ative humidity on the dates that imagery is collected 
for change detection purposes. When atmospheric con- 
ditions are not ideal, it may be necessary to a) delete 
some of the images from further consideration, and/or 
b) apply various atmospheric corrections to the remote 
sensor data so that spurious change due to atmospher- 
ic conditions is not identified (Jensen et al., 2012). 


Clouds: The most accurate image-to-image optical 
change detection is performed using aerial photogra- 
phy or digital satellite remote sensor data that are ac- 
quired with 0% cloud cover (e.g., Figure 12-2a,c of 
Charleston, SC). Clouds introduce extremely serious 
problems to change detection studies. Each cloud on a 
single image obscures the terrain below from investiga- 
tion (Figure 12-2b,d). It causes serious problems dur- 
ing the change detection because there is no land cover 
or structural information available where the cloud was 
present, resulting in ambiguous change detection in 
this area. All the user can do is create a mask and iden- 
tify certain areas as incomplete because of “cloud cov- 
er problems” (Jensen et al., 2012). 


Cloud shadow: In addition, almost every cloud casts 
a shadow that is visible in the imagery except when the 
cloud is at Nadir relative to the sensor system. Usually 
the cloud shadow is some angle away from the actual 
cloud in the image due to a) sensor viewing geometry 
conditions (e.g., the sensor is pointed 10° off-Nadir), 
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Clouds and Cloud Shadow in Landsat 8 Imagery of South Carolina 


Charleston 


a. Landsat 8 OLI 
Cloud free 
May 14, 2013 
Path = 16 Row = 37. 


b. Landsat 8 OLI 
30% Cloud cover 
May 30, 2013 
Path = 16 Row = 37. 


c. Landsat 8 OLI 
Cloud free 
May 17, 2014 
Path = 16 Row = 37. 


d. Landsat 8 OLI 
80% Cloud cover 
July 20, 2014 
Path = 16 Row = 37. 


FIGURE 12-2 a,c) Zero percent cloud cover on Landsat 8 scene Path 16/Row 37 on May 14, 2013 and May 17, 2014, respec- 
tively. These data are excellent for change detection purposes. b,d) 30% cloud cover on May 30, 2013 and massive 80% cloud 
cover on July 20, 2014. The use of these images for change detection would yield poor results (imagery courtesy of NASA). 


or b) the angle of the Sun which is a function of the 
time of year (e.g., June 21, the Summer Solstice) and 
the latitude of the study area (e.g., 30° N. Latitude). 
The terrain in the shadow area may be visible in the 
imagery, but it is usually much darker in tone due to 
the reduced amount of solar illumination reaching the 
terrain (Figure 12-2b,d). This causes the same feature 
(e.g., building, road, stand of trees) to have substantial- 
ly different spectral reflectance characteristics than the 
same feature observed on a different date (e.g., the next 
day) using the identical remote sensing data collection 
parameters with 0% cloud cover. When it is absolutely 
necessary to extract information from a geographic 
area that lies in cloud shadow, it is best to stratify this 
area (i.¢., mask it out) and classify it separately using 
its unique spectral properties. Once classified, the 
masked out area can be digitally stitched back into the 
final date dataset and then used in the change detec- 
tion procedure (Jensen et al., 2012). 


Relative Humidity and Other Atmospheric 

Constituents: Often the imagery used for change de- 
tection is free of clouds but there is substantial water 
vapor in the atmosphere resulting in high relative hu- 
midity or in certain areas there may be elevated 
amounts of haze or smog in the atmosphere. When this 
occurs, it may be necessary to apply atmospheric cor- 
rection to the individual dates of remote sensor data 
prior to change detection. This is done using either a) 
relative atmospheric radiometric normalization tech- 
niques (Jensen et al., 1995; Coulter et al., 2011), or b) 
absolute radiometric correction discussed in Chapter 6. 


Soil Moisture Conditions 


Ideally, the soil moisture in the study area is the same 
(held constant) on both of the images used to detect 
change. Unfortunately, this is often not the case. Fron- 
tal systems often move into an area and drop a rela- 
tively consistent amount of precipitation on the 
landscape. When this occurs, the entire area under in- 
vestigation might have similar soil moisture conditions. 
Conversely, sometimes thunderstorms drop precipita- 
tion over randomly located areas in a lobe-shaped pat- 
tern. This results in randomly located areas that have 
higher soil moisture than surrounding or adjacent ar- 
eas. Of course, the absence of soil moisture during 
drought conditions can also create a landscape that is 
extremely dry (Jensen et al., 2012). 


Geographic areas with extremely low soil moisture typ- 
ically will exhibit substantially higher spectral reflec- 
tance throughout the visible and _ near-infrared 
portions of the spectrum than the same terrain when it 
is saturated and has high soil moisture content. There- 
fore, multiple-date imagery of terrain that is either very 
moist or very dry can be problematic when subjected 
to change detection algorithms, resulting in spurious 
regions of change being identified due to differences in 
soil moisture rather than actual changes in features. If 
the two images under examination exhibit substantial 
differences in soil moisture, it may be possible to apply 
radiometric normalization to one of the images to 
make it have approximately the same spectral reflec- 
tance characteristics as the other image (e.g., Jensen et 
al., 1995; 2009). If only a portion of an image exhibits 
higher or lower than normal soil moisture conditions 


508 INTRODUCTORY DIGITAL IMAGE PROCESSING 


Phenological Cycle of Crops in California’s San Joaquin and Imperial Valleys and 
Landsat Multispectral Scanner Images of One Field During a Growing Season 
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FIGURE 12-3 Phenological cycles of a) sugarbeets, and b) cotton grown in the California San Joaquin Valley. Landsat MSS im- 
ages were obtained over a 12-month period. The color composite images (MSS RGB = bands 4, 2, 1) of two fields were ex- 


tracted and placed below the crop calendar information. 


due to thunderstorm precipitation or drought condi- 
tions, then it is best to stratify (mask-out) this area out 
and analyze it separately to extract land-cover infor- 
mation. This stratified area is then reinserted into the 
land-cover classification database prior to performing 
the change detection. 


Phenological Cycle Characteristics 


Natural ecosystems go through predictable cycles of 
development. Human beings often modify the land- 
scape in repeatable, predictable stages. These cycles of 
predictable development are often referred to as phe- 
nomenological or phenological cycles. Image analysts 
use these cycles to identify when remotely sensed data 
should be collected to obtain the maximum amount of 
usable change information. Therefore, analysts must be 
intimately familiar with the biophysical characteristics 
of the vegetation, soils, and water constituents of eco- 
systems and their phenological cycles. Likewise, it is 
imperative that they understand the phenological cy- 
cles associated with human-made development, such 
as residential expansion at the urban/rural fringe. 


Vegetation Phenology: Healthy vegetation grows 
according to relatively predictable diurnal, seasonal, 
and annual phenological cycles. Using approximately 
near-anniversary images greatly minimizes the effects 
of seasonal phenological differences that may cause 
spurious change to be detected in the imagery. For ex- 
ample, when attempting to identify change in agricul- 
tural crops using remote sensor data, the analyst must 
be aware of when the ground is prepared, when the 
crops are planted, when they reach maturity, and when 
they are typically harvested. Ideally, monoculture 
crops (e.g., corn, wheat, sugarbeets, cotton) are planted 
at approximately the same time of year. A month lag in 
planting date between fields of the same crop can cause 
serious change detection error. Second, the monocul- 
ture crops should be the same species. Different species 
of a crop can cause the crop to reflect energy different- 
ly on the multiple dates of anniversary imagery. In ad- 
dition, changes in row spacing and direction can have 
an impact. These observations suggest that the analyst 
must know the crop’s biophysical characteristics as well 
as the cultural land-tenure practices in the study area 
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Phenological Cycle of Cattails and Waterlilies in Par Pond in South Carolina 
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FIGURE 12-4 Annual phenological cycle of cattails and waterlilies in Par Pond, SC. Cattails are present throughout the year 
while waterlilies emerge in April and disappear in November. The best time to acquire imagery to discriminate between cat- 


tails and waterlilies is from mid-April to mid-May. 


so that the most appropriate remotely sensed data can 
be selected for change detection. 


For example, consider the phenological characteristics 
of two of the most important crops grown in the San 
Joaquin Valley in California shown in Figure 12-3. If 
we wanted to monitor the change in sugarbeet produc- 
tion during a growing season then we would have to ac- 
quire imagery from September through April. If we 
wanted to detect the change in sugarbeet production in 
multiple years, it would be wise to evaluate imagery 
collected from September through April. Conversely, if 
we wanted to monitor the change in cotton production 
through the season or in multiple years, we would ana- 
lyze imagery collected from May through October. 


Natural vegetation ecosystems such as wetland, forests, 
and rangeland have unique phenological cycles. For ex- 
ample, consider the phenological cycle of cattails and 
waterlilies found in lakes in the southeastern United 
States (Figure 12-4). Cattails persist year round in 
lakes and are generally found in shallow water adjacent 
to the shore (Jensen et al., 1993b). They begin greening 
up in early April and often have a full, green canopy by 
late May. Cattails senesce in late September to early 
October, yet they are physically present and appear 
brown through the winter months. Conversely, water- 
lilies and other nonpersistent species do not live 


through the winter. Waterlilies appear at the outermost 
edge of the cattails in mid-April and reach full emer- 
gence 6 to 8 weeks later. The waterlily beds usually per- 
sist above water until early November, at which time 
they die. 


The phenological cycles of cattails and waterlilies dic- 
tate the most appropriate times for remote sensing data 
acquisition. It is generally difficult to discriminate be- 
tween cattails and waterlilies from June through Au- 
gust because their spectral characteristics are very 
similar. This is also the time of maximum cloud cover 
and high humidity which reduces the availability of 
useful imagery. Conversely, from mid-April through 
mid-May, the cattails are well-developed while the wa- 
terlilies are just beginning to become established. This 
results in spectral reflectance differences between cat- 
tails and waterlilies as suggested by the red arrow in 
Figure 12-4. 


Many remote sensing change detection studies have 
failed because the analyst did not use accurate vegeta- 
tion phenological information when selecting the re- 
mote sensing imagery to be analyzed. 


Urban-Suburban Phenological Cycles: Urban and 
suburban landscapes also tend to have predictable phe- 
nological cycles. For example, consider the residential 
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Residential Development near Denver, CO, from October 8, 1976, to October 15, 1978 
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b. Panchromatic photograph 
obtained on October 15, 1978. 


FIGURE 12-5 a) Panchromatic aerial photograph of a portion of the Fitzsimmons 7.5-minute quadrangle near Denver, CO, on 
October 8, 1976. The original scale of the photography was 1:52,800. The land cover was visually photo-interpreted and clas- 
sified into 10 classes of residential developm ent using the logi c shown in Figure 12-6. b) Panchromatic aerial photograph of a 
portion of the Fitzsimmons 7.5-minute quadrangle on October 15, 1978. The original scale of the photography was 1:57,600. 
Comparison with the 1976 aerial photography reveals substantial residential land development. 


development from 1976 to 1978 in the 6-mi? portion of 
the Fitzsimmons 7.5-minute quadrangle near Denver, 
CO. Aerial photographs obtained on October 8, 1976, 
and October 15, 1978, reveal dramatic changes in the 
landscape (Figure 12-5). Most novice image analysts 
assume that change detection in the urban-rural fringe 
will capture the residential development in the two 
most important stages: rural undeveloped land and 
completely developed residential. Jensen (1981) identi- 
fied 10 stages of residential development taking place 
in this region based on evidence of clearing, subdivi- 
sion, transportation, buildings, and landscaping (Fig- 
ure 12-6). The remotely sensed data will most likely 
capture the terrain in all 10 stages of development. 
Many of these stages may appear spectrally similar to 
other phenomena. For example, it is possible that stage 
10 pixels (subdivided, paved roads, building, and com- 
pletely landscaped) may look exactly like stage 1 pixels 
(original land cover) in multispectral feature space if a 
relatively coarse spatial resolution sensor system such 


as the Landsat TM (30 30 m) is used. This can cause 
serious change detection problems. Therefore, the ana- 
lyst must be intimately aware of the phenological cycle 
of all urban phenomena being investigated, as well as 
the natural ecosystems. 


The dichotomous key described in Figure 12-6 was 
used to identify residential development in January 25, 
2007, and February 16, 2011 (near-anniversary date), 
high spatial resolution digital aerial photography of 
Beaufort County, SC (Figure 12-7). Once again, the 
land was completely cleared during the initial stages of 
development. Note the range of stages of development 
in the 2007 imagery including parcels with buildings in 
the process of being framed and parcels with just a 
foundation concrete pad (Figure 12-7a). Almost all of 
the residences in 2011 are completely landscaped (Fig- 
ure 12-7b). The 2011 imagery is overlaid with 2012 par- 
cel and street information. Each parcel with an address 
is highlighted with a flag icon (Figure 12-7b). 
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Progressive Stages of Residential Development 
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FIGURE 12-6 Dichotomous key used to identify progressive stages of residential development. Such development in Den- 

ver, CO, normally begins by clearing the terrain of vegetation prior to subdivision. In many geographic areas, however, some 
natural vegetation may be kept as landscaping. The absence or existence of natural vegetation dramatically affects the range 
of signatures that a parcel of land undergoes as it progresses from natural vegetation (1) to fully-landscaped residential hous- 


ing (10 or 17). 


Obscuration Considerations 


Sometimes the features of interest in a change detec- 
tion investigation such as buildings, roads, agricultural 
land, stands of trees, etc. are obscured from view on 
one or both of the images being analyzed by vegetation 
or intervening buildings. For example, Figure 12-8a is a 
nadir view of several single-family residences in Beau- 
fort, SC, collected on February 2, 2011. The image is 
centered on a home that is almost completely obscured 
from view by thick live oak canopy. The oblique views 
of the home looking from the North to the South (Fig- 
ure 12-8c) and from the East to the West (Figure 12-8e) 
also do not provide much information of value. Con- 
versely, the oblique views from the South to the North 
(Figure 12-8b) and from the West to the East 


(Figure12-8d) provide sufficient image information to 
determine that there is a residential home beneath the 
tree canopy. Note that the trees in the neighborhood 
also obscure some of the roads. Only a few companies 
such as Pictometry International, Inc. have software 
that can be used to detect change in buildings and 
roads from oblique imagery obtained on multiple 
dates. 


Effects of Tidal Stage on Change Detection 


Tidal stage is a crucial factor when conducting change 
detection in the coastal zone (Klemas, 2011). Ideally, 
the tidal stage is identical on multiple-date images used 
for change detection. Sometimes this severe constraint 
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Stages of Residential Development near Beaufort, SC, 
from January 25, 2007, to February 16, 2011 
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a. Stages of residential development identiXed in Pictometry 2007 imagery based on the 
dichotomous key in Figure 12-6 overlaid with 2012 Beaufort County, SC, parcel information. 
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b. Stages of residential development identiXed in 2011 imagery based on the 
dichotomous key in Figure 12-6 overlaid with Beaufort County parcels and street names. 


FIGURE 12-7 a) The stages of residential development described in Figure 12-6 applied to an area in Beaufort County, SC, in 
2007. b) The stages of residential development for the same geographic area in 2011. Many of the parcels in 2011 are subdi- 
vided, have access to paved roads, buildings are present, and the property is fully landscaped (i.e., residential development 
Stage 10). Each building with an icon has a Beaufort County residential address (imagery courtesy of Beaufort County GIS 


Department). 


can rule out the use of satellite remote sensing systems 
that cannot collect data off-nadir to meet the stringent 
tidal requirements. In such cases, the only way to ob- 
tain remote sensor data in the coastal zone that meets 
the stringent tidal requirements is to use suborbital 
sensors that can be flown at the exact time required. 
For most regions, images to be used for change detec- 
tion acquired at mean low tide (MLT) are preferred, | 
or 2 ft. above MLT are acceptable, and 3 ft. or more 


above MLT are generally unacceptable (Jensen et al., 
1993a; Klemas, 2011). 


Select the Most Appropriate Change 
Detection Algorithm 


The selection of an appropriate change detection algo- 
rithm is one of the most important considerations in the 


CHAPTER 12. CHANGEDETECTION 513 


Obscuration of Residential Buildings and Roads by Vegetation 


b. Oblique image looking from the South to the North 


d. Oblique image looking from the West to the East. 


e. Oblique image looking from the East to the West. 


FIGURE 12-8 a) Buildings, roads and other terrain may be obscured from view by tall dense tree canopy when imagery is col- 
lected at Nadir in leaf-on conditions. In this example the image is centered on the driveway of an obscured single-family resi- 
dence. b-e) The same residential area viewed from four different cardinal directions. The view from the South to the North (b) 
provides the most detailed information about the residential home (imagery courtesy of Beaufort County GIS Department). 


change detection process. There is no single change de- 
tection algorithm that is suitable for all change detec- 
tion projects (Chen et al., 2012). Digital change 
detection algorithms can provide binary land cover 
“change/no-change” information or more detailed the- 
matic “from-to” change information which identifies 


changes in land cover from forest, agriculture, etc., into 
another type of land cover such as residential housing, 
apartment complexes, roads, etc. (e.g., Lu et al., 2004; 
Im et al., 2009, 2012; Tsai et al., 2011). The following 
sections describe several of the most important change 
detection algorithms. 
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Analog On-screen Visualization Change Detection 
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FIGURE 12-9 The logic of analog visualization on a computer screen using two or three dates of registered remote sensor 
data placed in the RGB memory banks. Analysts can use a) individual original bands, and/or b) multiple-date transformations 
of the original data such as the Normalized Difference Vegetation Index (NDVI) (imagery courtesy of Richland County GIS 


Department). 


i Binary Change Detection 
ised Algorithms Provide “Change/ 
No-Change” Information 


a 


Two very useful binary change detection algorithms 
are: 1) analog “on-screen” visualization change detec- 
tion (Figure 12-9), and 2) binary change detecting using 
image algebra (e.g., band differencing and ratioing) or 
Principal Components Analysis (PCA) composite im- 
age analysis. 


Analog “On-Screen” Visualization 
Change Detection 


Analog on-screen visualization change detection typi- 
cally does not provide any quantitative change detec- 
tion information. Rather, change information is 
displayed on the computer screen in various colors for 
visual interpretation and understanding. 


Analog visualization involves the use of the three 
banks of graphics memory (RGB = red, green, and 


blue) located on the digital image processing system’s 
computer graphics card (refer to Chapter 3 for a dis- 
cussion about graphics memory). Basically, individual 
bands (or derivative products such as the Normalized 
Difference Vegetation Index—NDVI) from multiple 
dates of imagery are systematically inserted into each 
of the three memory banks (red, green, and/or blue) 
(Figure 12-9b) to highlight any changes in the imagery 
(Lunetta et al., 2006). 


For example, consider the 2004 (Date 1) and 2007 
(Date 2) digital aerial photography of an area in Bal- 
lentine, SC (Figure 12-10). The 2004 color-infrared 
color composite | X 1 ft. spatial resolution image ex- 
hibits complete canopy closure with little urban infra- 
structure present, except for a paved road and a few 
individual buildings (Figure 12-10a). The 2007 color- 
infrared image records a new residential subdivision 
(Figure 12-10b). The 2004 and 2007 near-infrared 
bands are shown in Figure 12-10c and d. A screen cap- 
ture of the analog visualization of the change between 
the two dates is displayed in Figure 12-10e. It was cre- 
ated by placing the 2004 near-infrared band in the red 


a. 2004 color-infrared | x 1 ft. color composite. 
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b. 2007 color-infrared 1 x 1 ft. color composite. 


a a 


d. 2007 near-infrared band. 


Analog Visualization 
Change Detection 


e. Analog visualization of change created by placing 
the 2004 NIR band in the red memory bank and the 
2007 NIR band in the green and blue memory banks. 


FIGURE 12-10 The use of analog visualization to detect changes in high spatial resolution imagery collected in 2004 and 
2007 of an area near Ballentine, SC (imagery courtesy of Richland County GIS Department). 


image processing video memory bank and the 2007 
near-infrared band in both the green and blue image 
processing video memory banks. The result is a strik- 
ing display of the residential change that has taken 
place in this area. 


Analog visualization may be used to visually examine 
virtually any type of registered, multiple-date imagery. 
For example, Figure 12-11 demonstrates how two dif- 
ferent types of multispectral data are analyzed to de- 
tect change. The Lake Mead, NV, watershed 
experienced severe drought from 2000 to 2003 resulting 
in a significant drawdown of the lake. Figure 12-lla 
depicts a color composite of Landsat ETM™ imagery 


collected on May 3, 2000, and Figure 12-11b is a color 
composite of ASTER imagery collected on April 19, 
2003. Both datasets record approximately the same 
near-infrared radiant flux and the images are within 14 
days of being anniversary dates. The ETM™ and AS- 
TER images were resampled to 30% 30 m (nearest- 
neighbor; RMSE + 0.5 pixel). Analog visualization 
was performed by placing the ASTER band 3 image in 
the red memory bank, the ETM’™ band 4 data in the 
green memory bank, and the ETM™ band 4 data in the 
green memory bank. This results in a clear depiction of 
the land exposed due to lake drawdown in shades of 
bright red. 


516 INTRODUCTORY DIGITAL IMAGE PROCESSING 


a. Landsat ETM+ data of Lake Mead, NV, 


obtained on May 3, 2000 (RGB = bands 4, 3, 2). 
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c. Analog visualization change detection where 
RGB = ASTER band 3, ETM* band 4, 
and ETM* band 4, respectively. 
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b. ASTER data obtained on April 19, 2003 
(RGB = bands 3, 2, 1). 


Analog Visualization Change Detection 
of Lake Mead, NV, using Landsat ETM* 
and ASTER Imagery 


FIGURE 12-11 a,b) Multiple-date images of Lake Mead, NV, collected by Landsat ETM* and ASTER. c) Analog visualization 
change detection using ASTER band 3 (0.76 — 0.86 Xm) and ETM* band 4 (0.75 — 0.90 Kim) (imagery courtesy of NASA). 


Esri, Inc., ChangeMattersfi 


Esri’s ChangeMattersfi Internet change detection pro- 
gram builds upon analog visualization logic to display 
change associated with multiple dates of remote sensor 
data, especially Landsat Multispectral Scanner (MSS), 
Thematic Mapper (TM), and ETM® imagery. Change- 
Mattersfi operates as a free Landsat viewer web service 
that is built on ArcGIS Server Image extension (Esri 
ChangeMatters, 2014). It calculates the change on the 


fly rather than storing the results that have been pre- 
processed. 


In December, 2008, the USGS made the entire Landsat 
archive, over 3 million images, available via the Inter- 
net at no cost (USGS, 2014). The USGS and NASA 
partnered to post the Global Land Survey (GLS) 
(http://landsat.usgs.gov/science_GLS.php) epochs of 
Landsat imagery, comprised of one image per each ep- 
och worldwide from the 1970s, 1990s, 2000, 2005, and 
2010. 


% 


a. 1975 Landsat MSS image. 


Esri ChangeMatters® Analysis of Mt. St. Helens from 1975 to 2010 


b. 2010 Landsat TM image. 
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FIGURE 12-12 Output from the Esri, Inc., ChangeMattersfi used to identify change on Mt. St. Helens from 1975 to 2010. 

a) A 1975 Landsat Multispectral Scanner (MSS) image (RGB = bands 4,2,1). b) A 2010 Landsat Thematic Mapper image (RGB 
= bands 4, 3, 2). c) A 2010 NDVI image was placed in the green graphics memory and a 1975 NDVI image was placed in the 

red and blue graphics memory. Unchanged areas appear in shades of grey because the earlier and later images should have 

approximately the same NDVI values. The NDVI change image highlights areas with vegetation decrease in magenta and veg- 
etation increase in light green (images courtesy of USGS and NASA). 


The ChangeMattersfi program dynamically mosaics 
images in different projections, formats, locations, pixel 
sizes, and time periods (Green, 2011). The server-based 
processing enables on-the-fly creation of multiple im- 
age products with no intermediate files being generat- 
ed. The application enables users to view different 
band combinations, different epochs of imagery, and 
to process multiple bands against one another to derive 
new products. Users can define what processing is to be 
performed on the imagery, and the server performs this 
directly on the source images, returning the informa- 
tion required for the area of interest. 


ChangeMattersfi displays are available in different 
standard band combinations including: near-infrared 
color composites (e.g., TM bands 4, 3, 2) useful for veg- 
etation studies and urban analysis; natural color with 
atmospheric penetration (e.g., TM bands 7, 4, 2) best 
suited for analysis of vegetation and some urban stud- 
ies; healthy vegetation; land/water boundary detection; 
and vegetation analysis providing information for agri- 
culture and forest management. Since these services are 
multitemporal, users can turn back the clock and easi- 
ly analyze how the landscape has changed in the region 
over the past 30 years (Green, 2011). 


We will use multiple dates of Landsat imagery of Mt. 
St. Helens, WA, to demonstrate the utility of the 
ChangeMattersfi change detection program. Mount 
St. Helens is an active stratovolcano located in Skama- 
nia County, WA. It is 154 km south of Seattle, WA, 
and 80 km northeast of Portland, OR. Mount St. Hel- 
ens is most famous for its catastrophic eruption on 
May 18, 1980, at 8:32 a.m. PDT. It is the deadliest and 
most economically destructive volcanic event in the 
history of the U.S. Fifty-seven people were killed; 250 
homes, 47 bridges, 24 km of railways, and 298 km of 
highway were destroyed. A massive debris avalanche 
triggered by an earthquake measuring 5.1 on the Rich- 
ter scale, caused an eruption, significantly reducing the 
elevation of the mountain’s summit from 9,677 ft. 
(2,950 m) to 8,365 ft. (2,550 m) and replacing it with a 
1 mile (1.6 km) wide horseshoe-shaped crater. The de- 
bris avalanche was up to 0.7 cubic miles (2.9 km?) in 
volume. The Mount St. Helens National Volcanic 
Monument was created to preserve the volcano and al- 
low for its aftermath to be scientifically studied. 


A 1975 Landsat Multispectral Scanner (MSS) image 
with an original spatial resolution of 79 X 79 m was se- 
lected to be the pre-eruption image (Figure 12-12a). 
The post-eruption image was a 30M 30 m Landsat TM 
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image collected in 2010 (Figure 12-12b). The change 
image (Figure 12-12c) is a composite of NDVIs ex- 
tracted from these two dates of imagery. The 2010 
NDVI image was placed in the green graphics memory 
bank and the 1975 NDVI image was placed in the red 
and blue graphics memory banks (Figure 12-12c) 
(Green, 2011). Unchanged areas appear in shades of 
grey because the earlier and later images should have 
approximately the same NDVI values. Areas with high- 
er NDVI in the later year than the earlier year are 
green indicating an increase in vegetation vigor or a de- 
crease in water level. Areas with lower NDVI in the lat- 
er year than the earlier year are magenta (red + blue = 
magenta) indicating a decrease in vegetation vigor 
(perhaps to urban land cover) or an increase in water 
level. Access to ChangeMattersfi on the Internet is free 
at http://changematters.esri.com/explore?tourid=10. 


Another ChangeMatters example is presented as the 
introductory image to this chapter. It consists of 1990 
and 2000 Landsat TM color-infrared color composites 
of the North Rim of the Grand Canyon. NDVI images 
were derived from these two TM datasets. The 2000 
NDVI image was placed in the green graphics memory 
bank, the 1990 NDVI image was placed in the red and 
blue graphics memory banks. The analog visualization 
highlights the forest fire burn scar present in the 2000 
Landsat TM image. 


Advantages of analog visualization change detection 
include the possibility of looking at two and even three 
dates of remotely sensed imagery (or derivative prod- 
ucts) at one time. Also, it is generally not necessary to 
atmospherically correct the remote sensor data used in 
analog visualization change detection. Unfortunately, 
the technique does not provide quantitative informa- 
tion about the amount of hectares changing from one 
land-cover category to another. Nevertheless, it is a 
useful analog method for qualitatively assessing the 
amount of change in a region, which might help with 
the selection of one of the quantitative change detec- 
tion techniques. 


Binary Change Detection Using 
Image Algebra 


The extraction of quantitative binary “change/no- 
change” information using two dates of imagery is one 
of the most heavily used change detection methods (Im 
et al., 2009, 2011; Klemas, 2011). Binary change be- 
tween two rectified images is typically performed using 
raster image algebra such as band differencing or band 
ratioing logic (Bruzzone and Prieto, 2000; Jensen and 
Im, 2007; Kennedy et al., 2009) or using a multiple- 
date composite image dataset and Principal Compo- 
nents Analysis (PCA) (Exelis, 2013; Rokni et al., 2015). 


Image Differencing Change Detection 


This technique involves subtracting band k of the Date 
2 image from band k of the Date | image where the two 
bands (k) ideally have the same spectral resolution 
(e.g., a near-infrared band = 700 — 900 nm) (Figure 12- 
13). If the two images (bands) have almost identical ra- 
diometric characteristics (i.e, the band k data have 
been normalized or atmospherically corrected), the 
subtraction ideally results in positive and negative val- 
ues in areas where radiance has changed and zero val- 
ues in areas of no change in a new change image 
(Figure 12-13a). 


The change image has a histogram with unique charac- 
teristics shown in Figure 12-13b (Im et al., 2011). 
When 8-bit data are analyzed, the potential range (do- 
main) of difference values found in the change image is 
—255 to +255 (Figure 12-13b and 12-14). The results 
can be transformed into positive values by adding a 
constant, c (e.g., 127). The operation 1s expressed as: 


ABV 25 = BVig (I) = BY, 2) ¥ ¢ (12.1) 


where 


A BV; ;, = change pixel value 
BV; ;, x1) = brightness value on date | 
BV; ; ;(2) = brightness value on date 2 
c =aconstant (e.g., 127) 
i= line number 
j = column number 
k =a single band (e.g., IKONOS band 3). 


The change image produced using image differencing 
usually yields a BV distribution approximately Gauss- 
ian, where pixels of no BV change are distributed 
around the mean and pixels of change are found in the 
tails of the distribution (Song et al., 2001). It is not 
necessary to add the constant c in Equation 12-1 if the 
image differencing output file is allowed to be floating 
point, i.e., the differenced pixel values can range from 
—255 to +255 (Figure 12-14a). Band ratioing involves 
the same logic, except a ratio is computed with values 
ranging from =~. to 255 and the pixels that did not 
change have a ratio value of | in the change image 
(Figure 12-14a). 


Image differencing change detection will be demon- 
strated using two datasets. The first example involves 
the Landsat ETM* imagery (band 4; 0.75 — 0.90 Xm) 
obtained on May 3, 2000, and the ASTER imagery 
(band 3; 0.76 — 0.86 Xm) of Lake Mead obtained on 
April 19, 2003 (Figure 12-15ab). These two images 
were differenced, resulting in the change image histo- 
gram shown in Figure 12-15c. Note that the change 
image histogram is symmetrical, suggesting that one 
image was histogram-matched (i.e., normalized) to the 
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Image Algebra Change Detection 


Date 1 


Date 2 


Frequency 


Advantages 
* Does not require atmospheric 


blue 
green 

RectiXed bands 
red 


NIR— 


blue 
green 

RectiXed bands 
red 


NIR: 


<+NIR 
+ NIR 


Composite layerstack 


a. Differenced or band-ratioed image 


b. Thresholded to produce binary 
“change/no-change” image 


c. “Change/no-change” image 


Disadvantages 
* No “from-to” change 


information available 
* Requires careful selection of the 
“change/no change” threshold 


correction, only histogram matching 


* Useful for identifying pixels that have 
changed in brightness value between dates 


FIGURE 12-13 The logic of image algebra change detection using image differencing or band ratioing (imagery courtesy of 


Richland County GIS Department). 


other (Kennedy et al., 2009). Most of the geographic 
area did not change between 2000 and 2003. Therefore, 
the vast majority of the pixels are found around the 
value 0 in the histogram. However, where the lake was 
drawn down, exposing bedrock, and where new vegeta- 
tion has grown on the exposed terrain has resulted in 
significant change documented in the change image in 
Figure 12-15d. There are actually two types of change 
in Figure 12-15d, bright green and red. All pixels in the 
change histogram below the first analyst-specified 


threshold were assigned red and all those above the 
second threshold were assigned green. 


Image Algebra Band Ratioing Change 
Detection 

Binary change detection using band ratioing involves 
dividing band k of the Date 1 image by band k of the 
Date 2 image where band k ideally has the same spec- 
tral resolution: 
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Binary Change Detection Using Statistical or Direct 
Symmetric/Asymmetric Approaches to Threshold Identification 


Pixel values that do not change 


User-specikled substantially between dates 
‘i change/no-change should hover about the mean 


lower threshold (LT) iesienneiiied 


change/no-change 


Pixels that upper threshold (UT) 

changed : 

> substantially : : 
B : 

5 between 

a. Histogram of Ss dates : 
change image. v : 
g g ia df 


0 : 
Band differencing —255 0 255 
Band ratioing 1/255 1 255 


Scaled to integer 0 127 255 


b. Statistical approach 
to threshold identiXcation. 


Band Differencing 
c. Symmetric direct e.g., 100 0 e.g., 100 
approach to threshold : : 
identiXcation. 

Band Ratioing 


d. Asymmetric direct 
approach to threshold 
identiXcation. 


LT UT 


FIGURE 12-14 A change image created from multiple date imagery using band differencing or band ratioing can be processed to 
identify change using b) statistical, c) symmetric, or d) asymmetric approaches (based on Figure 1 from Im, J., Lu, Z., and J. R. 
Jensen, 2011, “A Genetic Algorithm Approach to Moving Threshold Optimization for Binary Change Detection,” Photogrammetric 
Engineering & Remote Sensing, 77(2):167—180; and, Jensen et al., 2012). 
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Image Differencing Change Detection 


a. Landsat ETM* data of Lake Mead, NV, b. ASTER data obtained on April 19, 2003 
obtained on May 3, 2000 (RGB = bands 4, 3, 2). (RGB = bands 3, 2, 1). 


76814 


Change Image 
Histogram 


w= 1.981 
o = 37.44 


User-speciXed User-speciXed 
threshold threshold 


+ Red Green ———» 


0 


—208 (min) 0 255 (max) 


c. Histogram of the Moating point image created by d. Image differencing change detection. Red 
subtracting 2000 ETM? data from 2003 ASTER and green change pixels are associated with 
data. The symmetrical distribution conXrms that the thresholds located in the tails of the histogram. 


images were histogram-matched prior to processing. 


FIGURE 12-15 a) Landsat ETM* imagery of a portion of Lake Mead in Nevada obtained on May 3, 2000. b) ASTER data of 
Lake Mead obtained on April 19, 2003. c) The histogram of a change image produced by subtracting the ETM* 2000 band 4 
(0.75 — 0.90 Xm) data from the ASTER 2003 band 3 (0.76 — 0.86 Mm). d) Map showing the change as a function of the two 
thresholds identified in the change image histogram (original imagery courtesy of NASA Earth Observatory). 
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shi) 


= 12.2 
BV; ; x, 2) oe 


ABV; se 
Crippen (1988) recommended that all data used for 
multiple-date band ratioing be atmospherically cor- 
rected and free from any sensor calibration problems 
(e.g., a detector out of adjustment). If the two bands 
have almost identical radiometric characteristics, the 
resultant change image contains values that have a 
range (domain) from 1/255 to +255 (Figure 12-14a). 
Pixels that experienced minimal spectral change in 
band & on the two dates will have values that hover 
about the value 1.0. Unfortunately, the computation is 
not always simple because BV; ;; = 0 is possible in ei- 
ther date of imagery. Therefore, any BV; jk With a value 
of 0 is typically made equal to 1. Alternatively, a small 
value (e.g., 0.1) can be added to the denominator if it 
equals zero. 


To represent the range (domain) of the change image 
data in a standard 8-bit format (values from 0 to 255), 
normalizing functions are usually applied. Using these 
normalizing functions, the ratio value | is assigned the 
brightness value 127. Change image ratio values within 
the range 1/255 to | are assigned values from 1 to 127 
using the function (Jensen et al., 2012): 


ABV, 


Rik = Int[(B os 


x126)+1] (12.3) 


r 


Change image ratio values from >1 to 255 are assigned 
values within the range 128 to 255 using the function: 


ABV, 


oak (12.4) 


BY. , 
= Ini{ 128 + we ghe 
Values in the ratio change image near 127 typically ex- 
hibit minimal change, while pixel values in the tails of 
the distribution have usually changed significantly be- 
tween dates (Figure 12-14a). 


It is important to note that band ratioing or image dif- 
ferencing change detection does not have to be based 
on just the individual bands of remote sensor data. It 
may also be extended to comparing vegetation or other 
type of index information derived from multiple dates 
of imagery. For example, scientists often compute an 
NDVI on two dates and then subtract one from anoth- 
er to determine change (e.g., Lunetta et al., 2006; 
Green, 2011; Klemas, 2011): 


ANDVI,;= NDVI, ;(1)—NDVI,;(2)+¢ (12.5) 


It is not necessary to use the constant c in Equation 
12.5 if the image differencing output file is allowed to 
be floating point. The individual images used to ac- 
complish NDVI change detection should ideally be at- 
mospherically corrected. Change detection based on 


differencing multiple-date Kauth-Thomas transforma- 
tions (e.g., change in brightness, greenness, and/or wet- 
ness) have also been widely adopted (e.g., Franklin et 
al., 2002; Radke et al., 2005). 


Image Algebra Change Detection Using 
Statistical or Symmetric Thresholds 


An important consideration when performing either 
image differencing or band ratioing binary change de- 
tection is deciding where to place the lower threshold 
(LT) and upper threshold (UT) boundaries between 
“change” and “no-change” pixels associated with the 
change image histogram (Im et al., 2011; Patra et al., 
2013). The ideal threshold boundaries are rarely 
known a priori, but have to be determined. Sometimes 
a Statistical approach using a standard deviation from 
the mean is selected and tested such as +1 standard de- 
viation (Figure 12-14b) (Morisette and Khorram, 
2000). The statistical approach works well if change 
has occurred between dates in limited locations, e.g., 
<5% of the entire study site. If a larger portion of the 
study area has changed, the approach does not work 
well because the distribution of the pixel values is usu- 
ally skewed. It is also possible to directly place the 
thresholds a symmetric distance from the midpoint as 
shown in Figure 12-14c. 


Image Algebra Change Detection Using 
Asymmetric Thresholds 


The direct symmetric approach generally works well if 
the radiometric correction of the remotely sensed data 
is near-perfect. However, near-perfect radiometric cor- 
rection is rarely achieved due to factors described earli- 
er in this chapter. Consequently, as illustrated in Figure 
12-14d, asymmetric thresholds may work better than 
the symmetric thresholds for binary change detection 
using image differencing or band ratioing. Analysts of- 
ten prefer to experiment empirically, placing the 
thresholds at non-symmetric locations in the tails of 
the change image histogram distribution until a realis- 
tic amount of change is encountered (Figure 12-14d). 
Thus, the amount of change selected and eventually re- 
coded for display is often subjective and must be based 
on familiarity with the study area. 


Image Algebra Change Detection Using 
Moving Threshold Windows (MTW) 


Research on binary change detection has also focused 
on the development of automated methods of identify- 
ing the optimum lower and upper thresholds when cre- 
ating change images. For example, Im et al. (2009, 
2011) developed moving threshold windows (MTW) to 
identify the optimum location of the thresholds. The 
logic associated with the MTW-based approach to bi- 
nary change detection is illustrated in Figure 12-16. 
The approach consists of four basic processes. First it 


Binary Change Detection Using A 
Moving Threshold Window (MTW) Approach 


1. Determine the initial size of the MTW. 


No change 


[ee mean a : 
cr Center for UT 
symmetry 


2. Calibrate with the initial size of the MTW. 


}<—________—- Coverage 


+ Movement rate 


3. Calibrate with different sizes of MTW. 


4. Identify the optimum thresholds and size of the MTW 
resulting in the highest Kappa Coef&Icient of Agreement. 


FIGURE 12-16 The logic associated with using a moving 
threshold window to identify the optimum thresholds dur- 
ing binary change detection (based on Figure 1 in Im et al., 
2011, cited in Figure 12-14). 


is necessary to identify the initial size of the MTW. The 
analyst can specify the initial size or accept a default 
initial size that is set as the nearest value to the center 
for symmetry in the domain of a difference-typed 
change-enhanced image. The default initial size for a 
ratio-typed image is set as the farthest value from the 
center for symmetry in the domain. The next step in- 
volves locating the threshold window in the domain 
and evaluating the numerous locations of the window 
using an automated calibration model (Figure 12-16). 
Two parameters (i.e., coverage and movement rate of 
MTW) are determined by the analyst. The coverage 
represents where a threshold window can move and the 
movement rate is the step size with which the window 
moves in the coverage. Since only one (initial) size of 
threshold window is calibrated in the second process, 
different (i.e., increasing or decreasing) sizes of MTWs 
need to be evaluated to see if they produce better bina- 
ry change detection calibration. This is performed in 
the third process. The analyst can determine the maxi- 
mum (or minimum) size of MTW. The maximum (or 
minimum) value of the domain of the corresponding 
change-enhanced image is usually set as the default 
(Figure 12-16). 


Finally, the process identifies the optimum thresholds 
and size of MTW that result in the highest accuracy 
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(Figure 12-16). The MTW-based model calculates 
three accuracies including user’s and producer’s accu- 
racies for change, and the Kappa Coefficient of Agree- 
ment (Congalton and Green, 2009; Im et al., 2007, 
2009). Patra et al. (2013) provide additional approach- 
es to threshold selection when performing image differ- 
encing change detection. 


Multiple-Date Composite Image 
Change Detection 


Numerous researchers have rectified multiple dates of 
remotely sensed imagery (e.g., selected bands of two 
IKONOS scenes of the same region) and placed them 
in a single /ayerstack (Figure 12-17). This composite 
dataset can then be analyzed in a number of ways to 
extract change information (e.g., Deng et al., 2008; 
Kennedy et al., 2009; Rokni et al., 2015). 


Supervised and Unsupervised Classification of 
the Multiple-Date Composite Image to Detect 
Change 


First, a traditional classification using all n bands 
(eight in the example in Figure 12-17b) may be per- 
formed. Unsupervised classification techniques will re- 
sult in the creation of numerous clusters. Some of the 
clusters contain change information while others do 
not. The analyst must then use careful judgment to la- 
bel the clusters accordingly. 


Principal Components Analysis (PCA) 
Composite Image Change Detection 


PCA can be used to detect change in multiple dates of 
remote sensor data by merging the Date 1 and Date 2 
image datasets into a single multi-temporal composite 
image lJayerstack (Figure 12-17a). The data stored in 
the multispectral layerstack are usually highly correlat- 
ed with one another. The PCA evaluates the variance- 
covariance matrix associated with the n bands in the 
layerstack (yielding an un-standardized PCA) or the 
correlation matrix associated with the bands in the 
layerstack (yielding a standardized PCA). The PCA 
determines a new set of orthogonal axes that have their 
origin at the data mean and that are rotated so the data 
variance is maximized (Nielsen and Canty, 2008; Celik, 
2009; Kennedy et al., 2009; Almutairi and Warner, 
2010; Exelis, 2013). The output from a typical PCA is 
an uncorrelated set of m component images that ac- 
count for increasingly smaller amounts of variance 
(Figure 12-17c). 


The output component images can be analyzed using 
two different methods. First, a traditional classification 
using all n component images (six in the example in 
Figure 12-17d) may be performed. Unsupervised, su- 
pervised, or hybrid classification techniques are used to 
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Multiple Date Composite Image Change Detection 


Date I 


Date 2 


b. Traditional classification 
into n “from—to” classes 
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Frequency 


d. Traditional classification 
into n “from—to” classes 


Advantages 
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This produces a binary 
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Disadvantages 
* DifXcult to label change classes 
* Limited “from-to” information 


FIGURE 12-17 The logic of performing change detection using multiple dates of imagery placed in a layerstack. The layer- 
stack can then be analyzed using b) traditional unsupervised or supervised classification techniques, or c) by using Principal 
Components Analysis (PCA) (imagery courtesy of Richland County GIS Department). 


identify clusters in feature space associated with 
change (e.g., Rokni et al., 2015). The analyst must then 
label the clusters, which can be difficult (Deng et al., 
2008). This method does yield some “from-to” infor- 
mation if it can be deciphered. 


A more common approach is to select and analyze just 
the PCA component (i.e., band) that contains most of 
the important change information (Figure 12-17c,e). 
The first one or two PCA images generated from the 


multiple-date layerstack tend to account for variation 
in the image data that is not due to land-cover change. 
These are called stable components. When all of the 
bands are used, change information is often found in 
the third or fourth PCA band, but it varies depending 
on multiple-date scene content and the amount of 
change present between the two images. PCA bands 
that contain change information are called change 
components (Collins and Woodcock, 1996). The diffi- 
culty arises when trying to identify the ideal change 
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Principal Components Derived from a Multiple-Date Composite Image Dataset 


Lake Mead, NV, dataset consists of: 
* Landsat ETM? data obtained on May 3, 2000 (bands 2, 3, 4) 
* ASTER data obtained on April 19, 2003 (bands 1, 2, 3) 


d. Principal component 4. 


b. Principal component 2. 


e. Principal component 5. 


f. Principal component 6. 


FIGURE 12-18 Principal components derived from a multiple-date dataset consisting of Landsat ETM* and ASTER imagery. 
Principal component 2 contains change information. The first three principal components were placed in various memory 
banks to highlight more subtle changes, as shown in Figure 12-19 (original imagery courtesy of NASA). 


component image and extract change information (AI- 
mutairi and Warner, 2010). Once the PCA change 
component is identified, it is usually necessary to eval- 
uate its histogram and identify the “change versus no- 
change” thresholds in the tails of the distribution to 
isolate the change as previously discussed (Figure 12- 
17e). Change areas are typically highlighted as bright 
and/or dark pixels (Figure 12-17f). 


An example of a multiple-date composite image 
change detection will be demonstrated using two three- 
band Lake Mead, NV, datasets (Landsat ETM* and 
ASTER data). The datasets were merged into a single 


six-band dataset and subjected to a principal compo- 
nents analysis. This resulted in the creation of the six 
principal component images shown in Figure 12-18. 
Note that principal component 2 is a change compo- 
nent image (Figure 12-18b) containing detailed infor- 
mation about the area exposed by the lake drawdown. 
The histogram of the principal component 2 image in 
Figure 12-18b could easily be analyzed and a change 
map created (not shown). More subtle change infor- 
mation can be visually extracted from the multiple- 
date component dataset by placing each of the first 
three principal components in various memory banks, 
as shown in Figure 12-19. 
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Multiple-Date Composite Image Change Detection Based on 
Principal Components Viewed Using Analog Visualization 


a. Analog visualization change detection using 
RGB = principal components 3, 2, 1. 


c. Analog visualization change detection using 
RGB = principal components 2, 1, 3. 


b. Analog visualization change detection using 


RGB = principal components 1, 2, 3. 


Principal Component Images 
Derived from a Multiple-Date 
Composite Dataset Consisting of 


* Landsat ETM? data obtained on 
May 3, 2000 (bands 2, 3, 4) 


* ASTER data obtained on 
April 19, 2003 (bands 1, 2, 3) 


FIGURE 12-19 Multiple-date composite image change detection using PCA images that were created from a multiple-date 
composite dataset consisting of Landsat ETM* and ASTER data. Analog visualization was used to highlight the information 
content of the various principal component images. Principal components 2 and 3 contain much of the change information 


(original imagery courtesy of NASA). 


MDA Information Systems LLC, National Urban 
Change Indicator (NUCI)fi 


Another approach to the utilization of Landsat data 
for change detection is offered by MDA Information 
Systems LLC (MDA, 2014). Unlike the ChangeMat- 
ters application, NUCIfi utilizes both raster and vec- 
tor GIS data that have been preprocessed using MDA’s 
proprietary Correlated Land Change (CLC)fi_ process- 


es (Dykstra, 2012). The NUCI process systematically 
compares the spectral characteristics of Landsat data 
acquired over the past years to determine when a spe- 
cific 30 X 30 m pixel has experienced a “persistent” 
change. One of the output GIS data files includes the 
date when a change has occurred for each 30 X 30 m 
pixel in the entire contiguous 48 states. 
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MDAaA’s National Urban Change Indicator (NUCI® 
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FIGURE 12-20 a) MDA Information Systems, LLC's National Urban Change Indicator (NUCI) program uses registered Landsat 
(or other) data. b) Cross Correlation Analysis (CCA) is performed using imagery, and c) collateral geospatial data. d) Pattern 
recognition using CCA-derived and collateral data. e) Color-coded date of land-cover change based on NUCI analysis of 
multi-temporal Landsat data of Gaithersburg, MD. f) Enlargement with temporal change legend from 1987 to 2009 (adapted 
from the internet document, Dykstra, J., 2012, “Comparison of Landsat and RapidEye Data for Change Monitoring Applica- 
tions,” 11th Annual Joint Agency Commercial Imagery Evaluation Workshop, Fairfax, VA, April 17-19, 32 p). 


The input data for the Correlated Land Change pro- 
cess is a multi-temporal /ayerstack of co-registered 
Landsat multispectral scenes as shown in Figure 12- 
20a. MDA uses proprietary Cross Correlation Analysis 
(CCA) to determine pixel by pixel change by analyzing 
two-date change detection results calculated for a sys- 
tematic collection of pairwise permutations across a 
multi-temporal layerstack of precision co-registered 
Landsat multispectral data (Figure 12-20b). A multi- 
date pattern recognition process is then used to deter- 
mine and map urban and other anthropomorphic 
change on a pixel-by-pixel basis (Figure 12-20c-e). 
NUCI change can be reported in two classes: 


1. A 3-observation change is associated with pixels 
that remain in a consistent state (no measured 


change) for three consecutive dates in the multi- 
temporal layerstack; then change to a different 
state and remain in that different state for another 
three consecutive dates. This is determined by 
observing that each of the three observations 
within the new state showed, for that pixel, change 
with all three observation of the earlier state, while 
all pixels within a given state do not show change 
among themselves. 

2. A 2-observation change (called potential recent 
change) dataset contains pixels that exhibit a two- 
date change within the multi-temporal layerstack’s 
most recent five observations. A pixel is flagged as 
a potential recent change when two of the most 
recent observations of the multi-temporal image 
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TABLE 12-3 Change Attribute Majority (CAM) classes 
(adapted from Dykstra, J., 2012, cited in Figure 12-20). 


= 


. Probable Urban Change 


. Probable Water Change 


Probable River/Stream Change 


. Probable Shadow Change 


Probable Crop Change 


Change away from Roads 


Vegetated Probable Urban Change 


. Vegetated Probable Water Change 


ole|nloalataloelr 


. Vegetated Probable River/Stream Change 


10. Vegetated Probable Shadow Change 


11. Vegetated Probable Crop Change 


12. Vegetated Change away from Roads 


layerstack show no change with each other, and 
each showed change with a group of three earlier 
internally consistent (showing no change among 
the three) observations. Such a pixel could become 
a 3-observation change if a future observation 
measured no change with the upper pair within the 
temporal stack while, at the same time, showed 
change with each of the same three older 
observations. 
The Cross Correlation Analysis procedure uses an 
MDA-defined triply confirmed multi-temporal test to 
identify “persistent” change (Figure 12-20b). Most 
persistent change is associated with urban/cultural fea- 
tures. However, there are other phenomena that can 
trigger a persistent change (e.g., a change in reservoir 
level, recently cleared vegetation, strong seasonal shad- 
ows, etc.). Therefore, a collection of additional geospa- 
tial datasets is analyzed using GIS functions to “weed 
out” those pixels/polygons that are probably caused by 
non-urban phenomena (Figure 12-20c-e). 


Probable Urban Change pixels are those that pass 
tests based on GIS overlay analysis, such as: 


¢ Close to water but not close to NLCD-2006 defined 
urban, and 

¢ Not in an area of high local relief, and 

¢ Not coincident with USDA/NASS 2009 defined 
croplands, and 


* Not a large distance away from TIGER_2010 
$1400-level roads. 


The NUCI change data are available both as a raster 
and polygon representation of single or contiguous 
cells that have undergone change. These changes are 
categories representing the most probable cause of the 


NUCI detected change. The Change Attribute Major- 
ity classes are summarized in Table 12-3. 


The date and spatial distribution of land-cover change 
in Gaithersburg, MD, from 1987 to 2009 based on the 
National Urban Change Indicator (NUCI) analysis is 
shown in Figure 12-20f (Dykstra, 2012). Several agen- 
cies such as the Dept. of Defense, National Geospa- 
tial-Intelligence Agency, and the Dept. of Homeland 
Security make use of NUCI data. MDA Information 
Systems LLC will include Landsat 8 data into NUCI. 
Access to the NUCI database is by subscription. 


Continuous Change Detection and 
Classification (CCDC) using Landsat Data 


Remote sensing of the Earth has been conducted rou- 
tinely from some satellites for more than four decades 
(e.g., Landsat 1-8). Scientists are developing algo- 
rithms that can mine these multi-temporal datasets to 
identify the change taking place based on the use of ev- 
ery image available, instead of just two images at a 
time. For example, Zhu and Woodcock (2014) devel- 
oped a new algorithm for Continuous Change Detec- 
tion and Classification (CCDC) of land-cover based on 
the use of all available Landsat data. The algorithm is 
capable of detecting many kinds of land cover change 
continuously as new images are collected and provid- 
ing land-cover maps for any given time. Cloud and 
snow masking algorithms are used to eliminate “noisy” 
observations. The CCDC algorithm uses a threshold 
derived from all seven Landsat bands. When the differ- 
ence between observed and predicted images in a 
multi-temporal dataset exceeds a threshold three con- 
secutive times, a pixel is identified as land surface 
change. The actual land-cover classification is per- 
formed after change detection using a Random Forest 
Classifier. 


Zhu and Woodcock (2014) found that the CCDC re- 
sults applied to Landsat data of New England were ac- 
curate for detecting land surface change, with a 
producer’s accuracy of 98% and a user’s accuracy of 
86% in the spatial domain and a temporal accuracy of 
80%. The CCDC methodology is computationally in- 
tensive. Also, the algorithm requires a high temporal 
frequency of clear observations. For places with persis- 
tent snow or clouds, the CCDC model estimates may 
not be accurate as there are not enough observations 
to estimate the time series model. 


Similarly, Huang and Fried! (2014) developed a land- 
cover change detection algorithm that utilizes more 
than 10 years of MODIS data to identify changes in 
forest land cover. The method successfully identified 
pixels affected by logging and fire disturbance in tem- 
perate and boreal forest sites. 
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Preparation of a Block of Aerial Photography Using the Leica Photogrammetry Suite ® 


Photo 251 


Photo 253 


Display Mode 
@ Map Space 
© Image Space 
HB Image Extents 
i Image IDs 
Hi Control Points A 
BB Tie Points im 
O Check Points © 
Hi Point IDs 
OO Residuals 


Photo 251 


Stereo Model 


c. Anaglyph stereopair of 
photos #251 and #252 ready 
for feature extraction. 


FIGURE 12-21 a) Control associated w th a block of three vertical aerial photographs. b) Status of the preparation of the 
block of photography. c) Anaglyph stereo pair of photos #251 and #252 with the stereo model annotated (analyzed using Lei- 


ca Photogrammetry Suitefi software). 


ia Thematic “From-To” Change 
ied! Detection Algorithms 


While binary “change versus no-change” information is 
sufficient for many applications, it is much better to col- 
lect information on what the land use or land cover was 
and what it turned into. This is commonly referred to as 
obtaining thematic “from-to” change detection infor- 
mation. There are a variety of procedures and algo- 
rithms that can be used to extract “from-to” 
information from multiple-date remote sensor data. We 


will focus our attention on photogrammetric, LiDAR- 
grammetric, Post-Classification Comparison change 
detection using both per-pixel and object-based (OBIA) 
classifications, Neighborhood Correlation Image (NCI) 
Change Detection, and Spectral Change Vector Analy- 
sis change detection. 


Photogrammetric Change Detection 


Commercial photogrammetric engineering firms rou- 
tinely collect high spatial resolution stereoscopic aerial 
photography for public and commercial applications. A 
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Feature Extraction of Buildings and Road using ERDAS Stereo Analyst 
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a. Part of the 
stereo model 
under investigation. 


b. Feature extraction using the Koating mark 
(circled in yellow) while viewing the anaglyph 
stereo model of photos #251 and #252. 


c. Buildings and road shapelles extracted using soft-copy 
photogrammetry draped over photographs #251 and #252. 


FIGURE 12-22 a) A properly prepared stereo model can be analyzed using soft-copy photogrammetric techniques to extract 
very accurate building and road network information. Here only a small part of the stereo model is being analyzed. b) The op- 
erator uses the floating mark to map building footprints and roads. c) Overlay of the shapefile on top of the two aerial photo- 


graphs (analyzed using ERDAS Stereo Analystfi software). 


calibrated digital frame camera is typically used to col- 
lect the panchromatic, color and/or color-infrared digi- 
tal aerial photography. The photography is collected in 
systematic flightlines that usually have 60% to 80% end- 
lap and 20% sidelap (Jensen, 2007). Satellites such as 
World View-2 also collect stereoscopic imagery. 


Horizontal and/or vertical ground control points 
(GCPs) are obtained on the ground using in situ survey- 
ing in support of the data collection. These GCPs, addi- 
tional image-derived tiepoints, and onboard GPS/ 
inertial navigation information are used by the analyst 
to perform interior and exterior orientation to scale and 
level the stereoscopic models. Often, three-dimensional 
LiDAR data are used for control (Mitishita et al., 
2008). The digital aerial photography, camera calibra- 
tion information, and ground control are subjected to a 
rigorous aero-triangulation procedure that prepares the 
block of aerial photography for information extraction. 
For example, the block of three vertical aerial photo- 
graphs (#251, #252, and #253) shown in Figure 12-2la 
was prepared for stereoscopic analysis using interior 
and exterior orientation and aero-triangulation (Figure 
12-21b). Each of the resultant stereoscopic models) can 
be analyzed in three dimensions using anaglyphic glass- 
es. The stereo pair associated with photographs #251 
and #252 is shown in Figure 12-21c. Note the 60% 
overlap area that constitutes the stereoscopic model. 
Properly prepared stereoscopic aerial photography can 


be analyzed using photogrammetric mapping tech- 
niques to extract building footprints, road networks, 
vegetation, water distribution, etc. on individual dates 
of imagery (Elaksher, et al., 2003). For example, consid- 
er the small part of the stereo pair (photos #251 and 
#252) shown in Figure 12-22a that consists of single- 
family residential housing. The two photographs in the 
stereo model are projected onto the screen, one in blue 
and one in red. The analyst views the stereo model us- 
ing anaglyph glasses that have corresponding blue and 
green lenses. The mind fuses the two views of the terrain 
and creates a three-dimensional view of the terrain on 
the computer screen (Figure 12-22b). The analyst then 
uses a three-dimensional “floating mark” to identify 
features of interest (within the yellow circle in Figure 
12-22b). 


If a building footprint is desired, the analyst raises the 
floating mark above the terrain to the elevation of a ver- 
tex on the edge of the building roof. The floating mark 
is then moved around the perimeter of the roof to im- 
portant vertices where data are acquired. Upon comple- 
tion, the polygon is closed and represents the building 
footprint as if it were measured on the ground by a sur- 
veyor. Road network information is obtained by simply 
moving the floating mark so that it rests on the ground 
(i.e, ground elevation) and then clicking on points 
along the edge of the road (or the centerline of the road) 
while keeping the floating mark on the ground. Any 
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Photogrammetric Structural Change Detection 


a. Date | and Date 2 building footprints. 


b. Date 2 building footprint. 


FIGURE 12-23 a) Date 1 aerial photography with the building footprint extracted and outlined in yellow. The footprint of the 
same building derived from the Date 2 photography is displayed in red. b) Date 2 aerial photograph with the building foot- 
print extracted displayed in red (aerial photography courtesy of Pictometry International, Inc.). 


feature in the scene such as hydrographic features, 
driveways, and sidewalks can be mapped. All these geo- 
spatial features of interest are mapping in their exact 
planimetric (x, y) position (Figure 12-22c). 


Photogrammetric compilation is performed on the aeri- 
al photography obtained on Date 1. Thematic informa- 
tion such as buildings and roads that have not changed 
do not have to be recompiled on Date 2 aerial photog- 
raphy. Rather, only the new structures or roads are ex- 
tracted in the Date 2 photography and added to the 
database. For example, the Pictometry International 
(2014) ChangeFindrfi program automatically identifies 
changes in building footprints as demonstrated in Fig- 
ure 12-23. Such information can be used to determine if 
unauthorized building additions have been made. If de- 
sired, additional information can be extracted about 
other features such as the change from dirt to paved 
driveway and landscape modification. This type of 
three-dimensional data are ideal for subsequent use in 
urban change detection projects, e.g., to identify the 
change in city 3D building models (Qin, 2014). 


LiDARgrammetric Change Detection 


Commercial photogrammetric engineering firms also 
routinely collect high spatial resolution Light Detec- 
tion and Ranging (LiDAR) data for a variety of com- 
mercial and public applications (Figure 12-24). The 


LiDAR data are acquired using special instruments 
that transmit and receive hundreds of thousands of 
near-infrared (1,024 nm) pulses of laser energy every 
second (Jensen, 2007). Normally, only a few well- 
placed horizontal and/or vertical ground control 
points (GCPs) obtained using in situ surveying are re- 
quired when collecting LiDAR data for a given region. 
These GCPs and the onboard GPS/inertial navigation 
information are used to accurately locate the x, y, and 
z-position of every LiDAR first return, intermediate 
return, last return, and intensity return. These data are 
used to prepare digital surface models (DSMs) of the 
terrain that contain x, y, and z information about every 
tree, bush, building and road. These data can also be 
processed to prepare digital terrain models (DTMs) 
that contain only the bare Earth digital elevation infor- 
mation (Jensen, 2007). 


Properly prepared DSMs and DTMs can also be used 
to identify the location of individual buildings and 
road network using the individual Date 1 and Date 2 
LiDAR datasets. Building footprints and road net- 
works extracted on one date will be in the exact plani- 
metric location on the next date if the features have not 
changed. Buildings and roads or other features careful- 
ly extracted from Date 1 LiDAR data can be compared 
with buildings and roads and other features extracted 
from the Date 2 LiDAR data to identify changes (e.g., 
Sohn and Dowman, 2007). 
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Boston, MA, Building Footprints Extracted from LiDAR Data 


a. Building footprints for 
33 cities and towns in the 
metropolitan Boston area. 
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b. Enlargement of a small part of the metropolitan Boston study area. 


FIGURE 12-24 a) LiIDAR-derived building footprints for 33 cities and towns in metropolitan Boston, MA. The buildings are 
so dense in most areas that they appear dark gray at this small scale. b) An enlargement of a small part of the study area. 
Such LiDAR-derived information obtained on multiple dates is used routinely to monitor change (source of data: 


www.mass.gov/mgis/massgis.htm). 


Many cities, counties, and state departments of natural 
resources now routinely collect LiDAR data on a re- 
petitive basis and extract information from the data. 
For example, the Massachusetts Office of Geographic 
Information (MassGIS) extracted all of the building 
footprints for approximately 33 cities and towns in the 
metropolitan Boston area in 2002 (Figure 12-24a). An 
enlargement of a small part of the study area is shown 
in Figure 12-24b. It is now quite common for LiDAR- 
derived geospatial information to be used for change 
detection purposes. 


Post-Classification Comparison 
Change Detection 


Post-classification comparison change detection is one 
of the most widely used change detection methods 
(e.g., Ahlqvist, 2008; Warner et al., 2009; Griffiths et 
al., 2010; Tsai et al., 2011; Rokni et al., 2015). The 
method requires accurate geometric rectification of the 
two dates of imagery to a common map projection to 
within RMSE +0.5 pixel. The Date 1 image and the 
Date 2 image are classified using a) per-pixel analysis, 


or b) object-based image analysis (OBIA) (Gladstone 
et al., 2012). The classification schemes used for the 
Date 1 and Date 2 classification should be identical. A 
thematic change map is produced using a change de- 
tection matrix based on the use of a simple GIS overlay 
function (van Oort, 2007; Taubenbock et al., 2012). 


For example, the change detection matrix logic shown 
in Figure 12-25 can be used to highlight important 
change classes in the change thematic map, e.g., all of 
the pixels in the change image with a value of 9 were 
vegetation in the Date 1 classification and are now 
buildings in the Date 2 classification. Unfortunately, 
every classification error in the Date 1 and Date 2 clas- 
sification maps will also be present in the final change 
detection thematic map (Rutchey and Velcheck, 1994; 
Jensen et al., 1995). The minimum amount of change 
error in the change map is equivalent to the less accu- 
rate of the two individual classifications (van Oort, 
2007; Warner et al., 2009). Therefore, it is imperative 
that the individual date classification maps used in the 
post-classification change detection be as accurate as 
possible (Purkis and Klemas, 2011). 


CHAPTER 12. CHANGE DETECTION 533 


Post-Classification Comparison Change Detection 
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Per-pixel classification 
into 7 classes 
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Object-based (OBIA) 
classification into n classes 


Change Detection 
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Advantages 
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* Provides detailed “from-to” change 
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* Next base year map is already complete. 
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Disadvantages 
* Dependent on the accuracy of the 
Date | and Date 2 classiXcation maps. 
* Requires two separate classiXcations 
holding all class names constant. 


FIGURE 12-25 The logic of performing post-classification comparison change detection using multiple dates of imagery clas- 
sified using either a per-pixel or object-based image analysis (OBIA) classification and a change detection matrix tailored to 
meet the change detection information requirements (imagery courtesy of Richland County GIS Department). 


Per-Pixel Post-Classification Comparison 


An example of post-classification comparison change 
detection applied to per pixel classification is shown in 
Figures 12-26 and 12-27 using Landsat Thematic im- 
agery of an area centered on Fort Moultrie, SC, near 
Charleston. Nine land-cover classes were inventoried 
on each date (Figure 12-26). The 1982 and 1988 classi- 
fication maps were then compared on a pixel-by-pixel 
basis using an XM n GIS matrix algorithm whose logic 
is shown in Figure 12-27b. This resulted in the creation 
of a change image map consisting of brightness values 
from | to 81. The analyst then selected specific “from— 
to” classes for emphasis. Only a select number of the 72 


possible off-diagonal “from-to” land-cover change 
classes summarized in the change matrix (Figure 12- 
27b) were selected to produce the change detection 
map shown in Figure 12-27a. For example, all pixels 
that changed from any land cover in 1982 to Devel- 
oped Land in 1988 were color coded red (RGB = 255, 
0, 0) by selecting the appropriate “from-to” cells in the 
change detection matrix (10, 19, 28, 37, 46, 55, 64, and 
73). Note that the change classes are draped over a 
Landsat TM band 4 image of the study area to facili- 
tate orientation. Similarly, all pixels in 1982 that 
changed to Estuarine Unconsolidated Shore by De- 
cember 19, 1988 (cells 9, 18, 27, 36, 45, 54, 63, and 72), 
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Land-cover Classification of Fort Moultrie, SC, 
ee Landsat Thematic Mapper Data 


a SS 


c. Fort Moultrie, SC, on November 9, 1982. d. Fort Moultrie, SC, on December 19, 1988. 


Legend Estuarine Emergent Wetland 


Developed/Exposed Land Riverine Aquatic Beds 


Grassland Water 


Upland Forest Estuarine Unconsolidated Bottom 


Cultivated Land a Palustrine Forested Wetland 


RE 12-26 ab) Rectified Landsat Thematic Mapper data of an area centered on Fort Moultrie, SC, obtained on Novem- 
ber 9, 1982, and December 19, 1988 (RGB = bands 4, 3, 2). cd) Classification maps of the Fort Moultrie, SC, study area pro- 
duced from the November 9, 1982, and December 19, 1988, Landsat TM data. Some barren land is included in the 
Developed/Exposed Land category (original imagery courtesy of NASA). 


were depicted in yellow (RGB = 255, 255, 0). If desired, matrix can be used as an effective “from-to” change 
the analyst could highlight very specific changes such _— detection map legend. 

as all pixels that changed from Developed Land to Es- 
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Selected Change in Land Cover for Fort Moultrie, SC 


a. Fort Moultrie, SC. 


b. 
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FIGURE 12-27 a) Change detection map of the Fort Moultrie, SC, study areas derived from analysis of November 11, 1982, 
and December 19, 1988, Landsat Thematic Mapper data. The natures of the change classes selected for display are summa- 
rized in b) the “From—To” Change Detection Matrix (based on Jensen et al., 1993a). 


resolution imagery (Figure 12-28) (eg. Zhan et al., 
2005; Hofmann et al., 2008; Stow et al., 2011). Hur- 
skainen and Pellikka (2004) used OBIA to study the 
growth and change of informal settlements in South- 
east Kenya using scanned aerial photography from 
three different dates. The post-classification compari- 


son change detection method yielded high accuracy de- 
tecting new buildings. Moeller and Blaschke (2006) 
used multiple-date QuickBird imagery to detect new 
buildings in Phoenix, AZ. Matikainen et al. (2010) 
used LiDAR data and identified building changes us- 
ing an automated OBIA approach. They concluded 
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Post-classification Comparison Change Detection using Object-Based Image Analysis (OBIA) 


a. 2007 Bluffton, SC, 1 x 1 ft. imagery segmented to 40 scale. 


b. 2011 Bluffton, SC, 1 x | ft. imagery segmented to 40 scale. 


c. 2007 imagery segmented to 120 scale. 


d. 2011 imagery segmented to 120 scale. 


FIGURE 12-28 Post-classification comparison can be performed using per-pixel or object-based image analysis classification. 
This is an example of OBIA segmentation. Selected object-based segmentation of the Bluffton, SC, Pictometry International, 
2007 and 2011 natural color aerial photography at (a,b) 40 scale, and (c,d) 120 scale (imagery courtesy of Beaufort County GIS 


Department). 


that the main problem of building change detection 
was false detection due to new or demolished build- 
ings, and confusion with adjacent (background) fea- 
tures. Tsai et al. (2011) delineated new buildings by 
combining OBIA approaches using multi-date high 
spatial resolution satellite data. Buildings constructed 
between 2002 and 2010 in Accra, Ghana, were delin- 
eated and quantified. Taubenbock et al. (2012) used 
decision-tree classification and post-classification com- 
parison to detect change in very large cities (>10 mil- 
lion people). Chen et al. (2012) provided an overview 


of object-based change detection logic and consider- 
ations. Gartner et al. (2014) used OBIA to extract tree 
crown area and diameter in multiple dates of satellite 
imagery and then compared the diameter of the crowns 
to detect change. 


An example of post-classification comparison change 
detection using OBIA applied to 2007 and 2011 Pic- 
tometry, Inc., 1 X 1 ft. spatial resolution digital aerial 
photography of an area in Bluffton, SC, is shown in 
Figures 12-28 and 12-29. The 2007 and 2011 aerial 
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Post-classification Comparison Change Detection using Object-Based Image Analysis (OBIA) 


a. 2007 Bluffton, SC, classiXcation map. 


c. Union of 2007 and 2011 classiXlcation maps 
with important classes highlighted. 


b. 2011 Bluffton, SC, classiication map. 
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FIGURE 12-29 Post-classification change detection of Bluffton, SC, based on 2007 and 2011 Pictometry, Inc., natural color 
aerial photography and object-based image analysis. a) Classification of 2007 photography (rasterized). b) Classification of 
2011 photography (rasterized). c) Union of 2007 and 2011 classification maps using GIS map overlay function with selected 


“from-to” classes highlighted. 


photography was segmented to the following “scales” 
using eCognition OBIA: 40, 50, 60, 70, 80, 90, 100, 
110, and 120. Only the 40- and 120-scale segmenta- 
tions are shown in Figures 12-28ab and 12-28cd, re- 
spectively. The 2007 aerial photography was classified 
using the coarse 120 scale segmentation level because 
this level was sufficient to capture the spatial and spec- 
tral detail of the land cover present on this date (Figure 
12-29a). Conversely, the 2011 photography was classi- 
fied using the more detailed 40-scale segmentation be- 
cause it was necessary to identify the vegetation and 


shadows between the individual residential houses 
(Figure 12-29b). 


The post-classification change detection map was cre- 
ated by analyzing the 2007 and 2011 OBIA classifica- 
tion maps (converted to raster format) using a GIS 
overlay “union” function, resulting in 36 possible 
“from-to” change classes shown in Figure 12-29. Sev- 
eral important “from-to” change classes are highlight- 
ed in Figure 12-29c: from vegetation to building (red), 
from vegetation to road (yellow), and from vegetation 
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Change Detection Based on Neighborhood Correlation 
Image Analysis and Decision-tree Classification 


Date 1 Image 


Date 2 Image 


Create Multi-level Neighborhood Correlation Images (NCIs) using: 
«A circular or rectangular neighborhood, e.g., 3 x 3 pixels 


Extract Training/Test Data from Multi-date Composite Dataset: 
* Muli-date imagery + NCI images (correlation, slope, aspect) 


Decision-tree Classification of Multi-date Composite Dataset: 
¢ Muli-date imagery + NCI images (correlation, slope, aspect) 


Accuracy Assessment 


FIGURE 12-30 Steps required to perform change detection 
using Neighborhood Correlation Image (NCI) analysis and 
decision-tree classification (based on Figure 2 in Im and 
Jensen (2005) cited in Figure 12-31). 


to driveway (orange). Roads and water present in both 
dates of photography are shown in black and cyan, re- 
spectively, for illustrative purposes. Note that the geo- 
metric registration between dates was very good as the 
narrow golf cart paths and the major roads exhibited 
only a slight amount of error along the edges. 


Post-classification comparison change detection is 
widely used and easy to understand. When conducted 
by skilled image analysts, it represents a viable tech- 
nique for the creation of change detection products. 
Advantages include the detailed “from-to” informa- 
tion that can be extracted and the fact that the classifi- 
cation map for the next base year is already complete 
(Arzandeh and Wang, 2003; Jensen et al., 2012). How- 
ever, the accuracy of the change detection depends on 
the accuracy of the two separate classification maps. 


Neighborhood Correlation Image 
(NCI) Change Detection 


Neighborhood Correlation Image (NCI) Change De- 
tection is based on the creation of a new three-channel 
NCI that contains three unique types of information 
(Im and Jensen, 2005; Im et al., 2007, 2008): 


* correlation, 

¢ slope, and 

* intercept 

that can be related to the change in multiple-date imag- 
ery (Figure 12-30). The correlation information de- 


rived from a specific neighborhood of pixels (e.g., 3 X 
3) contains valuable change information associated 


with a central pixel and its contextual neighbors. Slope 
and intercept images provide change-related informa- 
tion that can be used as ancillary data to facilitate 
change detection with correlation. The degree of corre- 
lation, slope, and intercept can then be used to produce 
detailed “from-to” change information when com- 
bined with expert system (e.g., decision tree) or object- 
oriented classification techniques. 


NCI analysis is based on the change magnitude and di- 
rection of brightness values by band in a specific neigh- 
borhood between two multispectral remote sensing 
datasets. If the spectral changes of the pixels within a 
specified neighborhood between two image dates are 
significant, the correlation coefficient between the two 
sets of brightness values in the neighborhood will fall 
off to a lower value. The slope and intercept values 
may increase or decrease depending on the magnitude 
and direction of the spectral changes. Ideally, if there is 
no change, pairs of brightness values between the two 
dates of imagery should be located along y = x, and be 
located far from y = x if change has occurred between 
the two image dates. Consequently, correlation coeffi- 
cients are lower when change occurs and higher when 
only a small amount of change is present. Intermediate 
correlation values (e.g., r = 0.7) are generally encoun- 
tered in areas that have changed only slightly (e.g., wid- 
ened roads, expanded buildings) or changed vegetation 
status (e.g., the same vegetation captured in a slightly 
different phenological cycle). 


Correlation analysis is used to detect change in rela- 
tively small local neighborhoods (e.g., 3 M 3 pixels) in 
two registered remote sensor images obtained on dif- 
ferent dates. The correlation between the brightness 
values of two dates of imagery in the local neighbor- 
hoods is calculated using the equations (Im and Jen- 
sen, 2005; Im et al., 2007): 


COV 
p= (12.6) 
S159 
where 
n 
> (BV, _ U1 )(BV 5 - U5) 
cov, = —=L—__________ (127) 


n-1 


and r is Pearson’s product-moment correlation coeffi- 
cient, cov, is the covariance between brightness values 
found in all bands of the two date datasets in the neigh- 
borhood, and s; and s» are the standard deviations of 
the brightness values found in all bands of each dataset 
in the neighborhood, respectively. BV;,; is the ith 
brightness value of the pixels found in all bands of im- 
age | in the neighborhood, BV; is the ith brightness 
value of the pixels found in all bands of image 2 in the 
neighborhood, n is the total number of the pixels 


Images used in Neighborhood Correlation Image Analysis Change Detection 


a. Near-infrared band obtained on Sept. 23, 1999. 


CHAPTER 12. CHANGEDETECTION 539 


> ‘ > é 


b. Near-infrared band obtained on Oct. 10, 2000. 


FIGURE 12-31 Near-anniversary multiple date (1999 and 2000) near-infrared imagery of an area in Edisto Beach, SC, at 

1 1 ft. spatial resolution. New residential housing that is completely built is highlighted in red. New cleared land is 
highlighted in yellow (based on Figure 1 in Im, J., and J. R. Jensen, 2005, “Change Detection Using Correlation Analysis and 
Decision Tree Classification,” Remote Sensing of Environment, 99:326-340). 


found in all bands of each dataset in the neighbor- 
hood, and y, and yu, are the means of brightness val- 
ues found in all bands of the two date (1 and 2) images 
in the neighborhood, respectively. 


Slope (a) and y-intercept (b) are calculated using the 
following equations from the least squares estimates: 


cov 
a= = (12.8) 
Sd 


b= tel i 
nN 


n n 
», BV jy-a BV, 
=1 (12.9) 


Figure 12-30 summarizes the steps required to imple- 
ment the change detection based on neighborhood 
correlation image analysis and decision tree classifica- 
tion, including: 


* Geometrically and radiometrically preprocess the 
multi-temporal remote sensor data (previously dis- 
cussed). 


* Create the multi-level Neighborhood Correlation 
Images (NCIs), i.e., correlation, slope, and intercept 
images (bands) based on the neighborhood configu- 
ration specified. 


* Collect training/test data from the multi-date, multi- 
channel dataset (e.g., eight spectral bands plus 
multi-level NCI features: correlation, slope, and 
intercept). 

* Use the training/test data to generate the production 
rules to extract “from-to” change information 
based on decision-tree logic classification and create 
a change detection map. 


¢ Perform an accuracy assessment to determine the 
effectiveness of the change detection model. 


Digital frame camera imagery of a residential area in 
Edisto Beach near Charleston, SC, is used to demon- 
strate the utility of Neighborhood Correlation Images 
for change detection (Figure 12-3lab). The data were 
obtained on September 23, 1999, and October 10, 
2000, in four spectral bands (blue, green, red, and near- 
infrared). Both dates of imagery were geo-referenced 
to a Universal Transverse Mercator projection with an 
RMSE < 0.5 pixels. The data were radiometrically nor- 
malized using water and bare soil radiometric ground 
control points identified as pseudo-invariant pixels us- 
ing techniques described in Chapter 6. Regression 
equations allowed the individual September 23, 1999, 
multispectral bands to be radiometrically normalized 
to the October 10, 2000, multispectral bands. Correla- 
tion, slope, and intercept images derived from the 
multi-date images are displayed in Figure 12-32. 
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c. Intercept image. 


b. Slope image. 


Neighborhood Correlation Images (NCI) 
Derived from Multiple-Date Imagery 
* Correlation 
* Slope 
* Intercept 


FIGURE 12-32 Three Neighborhood Correlation Images (NCI) derived from multiple-date imagery including a) correlation, 
b) slope, and c) intercept (based on Figure 4 in Im, J., and J. R. Jensen, 2005, “Change Detection Using Correlation Analysis 
and Decision Tree Classification,” Remote Sensing of Environment, 99:326-340). 


The correlation coefficient (r) indicates the spectral 
change between the two images on a pixel by pixel ba- 
sis. However, high correlation does not always guaran- 
tee that change has not occurred. For example, Figure 
12-33 displays the results of examining just three loca- 
tions within the 1999 and 2000 high spatial resolution 
images. Sample A is from an unchanged area (Devel- 
oped on both dates). Samples B and C are from 
changed areas (grass to barren, and barren to devel- 


oped, respectively). All three locations exhibit high 
correlation (i.e., r > 0.8). Thus, locations B and C are 
not separable from the unchanged area based solely on 
correlation coefficient information with a threshold of 
r = 0.8. However, when we consider the slope and y- 
intercept of the regression lines shown in Figure 12-33, 
we find that B and C are from changed areas. Ideally, a 
pixel from an unchanged area will have a slope of ~1 
and a y-intercept near 0 similar to sample A shown in 


CHAPTER 12. CHANGE DETECTION 541 


250 


Correlation coefficient > 0.8 


id y= 1.7516x + 24.635 


y = 0.9526x— 3.5164 
150 


100 


Brightness value in 2000 image 


y= 0.4596x— 7.6911 


50 
= Changed (B > D) 
A Changed (G > B) 
@ No Change (D) 


0 50 100 150 200 250 
Brightness value in 1999 image 


FIGURE 12-33 Correlation results associated with the Edisto, SC, multiple-date (1999 and 2000) imagery (based on Figure 3 
in Im, J., and J. R. Jensen, 2005, “Change Detection Using Correlation Analysis and Decision Tree Classification,” Remote 
Sensing of Environment, 99:326-340). 


Change Detection Based on Correlation Image Analysis and Decision-Tree Classification 


To 
Barren 2000 


(Bare Soil, 


Developed Transitional) Tree Grass Shadow 


Developed 


Barren 
(Bare Soil, 
Transitional) 


From 
1999 


Tree 


Change in Grass 


b. Change detection matrix. 


a. Change classiXcation with NCIs based on 
a circle neighborhood with a 3-pixel radius. 


FIGURE 12-34 “From-to” change information based on Neighborhood Correlation Image (NCI) analysis and decision-tree 
classification (based on Figure 8 in Im, J., and J. R. Jensen, 2005, “Change Detection Using Correlation Analysis and Decision 
Tree Classification,” Remote Sensing of Environment, 99:326-340). 
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Spectral Change Vector Analysis 
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FIGURE 12-35 a-d) Schematic diagram of the Spectral Change Vector Analysis change detection method. e) Possible 
change sector codes for a pixel measured in three bands on two dates. 


Figure 12-33. Conversely, change areas like B and C 
have relatively higher or lower slopes and y-intercepts. 
In this example, location B has a slope of ~1.8 and its 
y-intercept is ~25. Location C has a slope of < 0.5 and 
a negative y-intercept. Consequently, locations A, B, 
and C can be distinguished as being change or no- 
change by incorporating slope and intercept data in 
addition to correlation information. 


Decision-tree logic can be used to classify the multi- 
date composite image datasets consisting of two dates 
of image data plus the three NCI features (correlation, 
slope, and intercept). Rules generated from a C5.0 de- 
cision tree were used in an expert system to identify 
“from-to” change information on a pixel by pixel basis 
(Figure 12-34). 


Spectral Change Vector Analysis 


When land undergoes a change or disturbance between 
two dates, its spectral appearance normally changes. 
For example, consider the red and near-infrared spec- 
tral reflectance characteristics of a single pixel dis- 
played in two-dimensional feature space (Figure 12- 
35a). It appears that the land cover associated with this 
particular pixel has changed from Date | to Date 2 be- 
cause the pixel resides at a substantially different loca- 
tion in the feature space on Date 2. The vector 
describing the direction and magnitude of change from 


Date | to Date 2 is a spectral change vector (Malila, 
1980; Warner et al., 2009; Carvalho et al., 2011; Kon- 
toes, 2013; Yuan et al., 2015). 


The total change magnitude per pixel (CMpjx¢]) is com- 
puted by determining the Euclidean distance between 
endpoints through n-dimensional change space (Mi- 
chalek et al., 1993; Kontoes, 2013): 


n 


2 
= [BV jx (date2) =f V iik(datel)! (12.10) 
k=1 


CM 


pixel — 


where BV; ; ¢(dater) and BV; ; K(date1) are the Date | and 
Date 2 pixel values in band k. 


A scale factor (e.g., 5) can be applied to each band to 
magnify small changes in the data if desired. The 
change direction for each pixel is specified by whether 
the change is positive or negative in each band. Thus, 
2” possible types of changes can be determined per pix- 
el (Virag and Colwell, 1987). For example, if three 
bands are used there are 2? or 8 types of changes or 
sector codes possible as shown in Table 12-4. To dem- 
onstrate, consider a single registered pixel measured in 
just three bands (1, 2, and 3) on two dates. If the 


change in band 1 was positive (e.g., ee iid \(date 2) = 45; 
BV‘; (date 1) = 38; BV change = 45 — = 7), and the 
change in band 2 was positive (e.g., BV acne = 20; 


TABLE 12-4 Sector code definitions for change vector anal- 
ysis using three bands (+ indicates pixel value increase from 
Date 1 to Date 2; — indicates pixel value decrease from Date 


1 to Date 2) (Michalek et al., 1993). 
Change Detection 

Sector 

Code Band 1 Band 2 Band 3 
1 = _ _ 
Z - - + 
3 - + - 
4 - + +r 
5 + - - 
6 + - + 
7 + + - 
8 + + + 


BV‘ 2(datel) = = 10; BV change = 20-10 = 10), and the 
change in band 3 was negative (e.g., rie (i,j 3(date2) = = 25; 
BV; ij,3(datel) — = 30; BV chang g=2a> = —5), then. the 
change magnitude of the pixel a =“ CM pixel = =7T+ 
10° =57 = 174, and the change sector code for this pixel 
would be “+, +, —” and have a value of 7, as shown in 
Table 12-4 and Figure 12-35e. For rare instances when 
pixel values do not change at all between the two dates, 
a default direction of + may be used to ensure that all 
pixels are assigned a direction (Michalek et al., 1993). 


Change vector analysis outputs two geometrically reg- 
istered files; one contains the sector code and the other 
contains the scaled vector magnitudes. The change in- 
formation may be superimposed onto an image of the 
study area with the change pixels color-coded accord- 
ing to their sector code. This multispectral change 
magnitude image incorporates both the change magni- 
tude and direction information (Figure 12-35a). The 
decision that a change has occurred is made if a thresh- 
old is exceeded (Virag and Colwell, 1987). The thresh- 
old may be selected by examining deep-water areas (if 
present), which should be unchanged, and recording 
their scaled magnitudes from the change vector file. 
Figure 12-35b illustrates a case in which no change 
would be detected because the threshold is not exceed- 
ed. Conversely, change would be detected in Figures 
12-35c and d because the threshold was exceeded. The 
other half of the information contained in the change 
vector, that is, its direction, is also shown in Figure 12- 
35c and d. Direction contains information about the 
type of change. For example, the direction of change 
due to clearing should be different from change due to 
regrowth of vegetation. 
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Change vector analysis has been applied successfully 
to forest change detection in northern Idaho (Malila, 
1980), mangrove and reef ecosystems along the coast 
of the Dominican Republic (Michalek et al., 1993), 
and agriculture in Brazil (Carvalho et al., 2011). It is 
the change detection algorithm of choice for producing 
the MODIS Vegetative Cover Conversion (VCC) prod- 
uct being compiled on a global basis using 250 m sur- 
face reflectance data (Zhan et al., 2002). The method is 
based on measuring the change in reflectance in just 
two bands, red (Ap ;eq) and near-infrared (Ap ,i,), be- 
tween two dates and using this information to compute 
the change magnitude per pixel, 


2 


2 
M pixel ~ (Pea) + (AP air) (12.11) 
and change angle (0): 
AP red 
D aieel = arctan oa (12.12) 


The change magnitude and angle information is then 
analyzed using decision-tree logic to identify impor- 
tant types of change in the multiple-date MODIS im- 
agery (Zhan et al., 2002). Chen et al. (2003) developed 
an improved change vector analysis methodology that 
assists in the determination of the change magnitude 
and change direction thresholds when producing land- 
cover change maps. Carvalho et al. (2011) developed 
improved methods of computing the spectral direction. 
The chief advantage of this approach is that it gener- 
ates a single image of change information insensitive to 
illumination variation. Kontoes (2013) developed an 
operational version of change vector analysis. 


Change Detection Using an Ancillary 
Data Source as Date 1 


Sometimes there exists a land-cover data source that 
may be used in place of a traditional remote sensing 
image in the change detection process. For example, 
the U.S. Fish and Wildlife Service conducted a Nation- 
al Wetland Inventory (NWI) of the U.S. at 1:24,000 
scale. Much of these data have been digitized. Instead 
of using a remotely sensed image as Date | in a coastal 
change detection project, it is possible to substitute the 
digital NWI map of the region (Figure 12-36). In this 
case, the NWI map is recoded to be compatible with 
the classification scheme being used. Next, Date 2 of 
the analysis is classified and then compared on a pixel- 
by-pixel basis with the Date 1 information using post- 
classification comparison methods. Traditional “from— 
to” information can then be derived. 
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Change Detection Using an 
Ancillary Data Source as Date 1 


Ancillary data source, 
Date 1 e.g., National Wetlands 
Inventory (NWI) map 
blue 


green RectiXed remote 
red sensor data 


NIR 


7 Classication map of Date 2 


ClassiXcation map of Date | 


1) Perform post-classiXlcation 
comparison change detection. 
or 

2) Update Date | NWI map with 
Date 2 change information 
using GIS dominate function. 


Advantages 

* May reduce change detection 
errors (omission and commission) 

* Provides “from-to” change 
class information 

* Requires a single classiXication 


Disadvantages 

* Depends on quality 
of ancillary 
information 


FIGURE 12-36 Diagram of change detection using an an- 
cillary data source as Date 1 (imagery courtesy of Richland 
County GIS Department). 


Advantages of the method include the use of a well- 
known, trusted data source (e.g., NWI) and the possi- 
ble reduction of errors of omission and commission. 
Detailed “from-to” information may be obtained us- 
ing this method. Also, only a single classification of the 
Date 2 image is required. It may also be possible to up- 
date the NWI map (Date 1) with more current wetland 
information (this would be done using a GIS dominate 
function and the new wetland information found in the 
Date 2 classification). The disadvantage is that the 
NWI data must be digitized, generalized to be compat- 
ible with a classification scheme, and then converted 
from vector to raster format to be compatible with the 
raster remote sensor data. Manual digitization and 
subsequent conversion may introduce error into the 
database, which may not be acceptable (Lunetta et al., 
1991). 


Kennedy et al. (2009) point out that this particular 
change detection method cannot be extended indefi- 
nitely. Land-cover classes in the original map are con- 
tinually degraded by change, reducing their spectral 
fidelity over time and requiring eventual creation of a 
new land-cover map. 


Change Detection Using a 
Binary Change Mask Applied to Date 2 
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class information 


FIGURE 12-37 Diagram of change detection using a binary 
change mask applied to Date 2 (imagery courtesy of Rich- 
land County GIS Department). 


Change Detection Using a Binary 
Change Mask Applied to Date 2 


This method of change detection is very effective. First, 
the analyst selects the base image referred to as Date | 
at time n. Date 2 may be an earlier image (n—1) or a 
later image (n + 1). A traditional classification of Date 
1 is performed using rectified remote sensor data. 
Next, one of the bands (e.g., band 2 in Figure 12-37) 
from both dates of imagery is placed in a new dataset. 
The two-band dataset is then analyzed using various 
image algebra change detection functions (e.g., band 
ratio, image differencing) to produce a new change im- 
age file. The analyst selects a threshold value to identify 
areas of “change” and “no change” in the new image, 
as discussed in the section on image algebra change de- 
tection. The change image is then recoded into a bina- 
ry mask file consisting of areas that have changed 


between the two dates. Great care must be exercised 
when creating the “change/no-change” binary mask 
(Jensen et al., 1993a).The change mask is then overlaid 
onto Date 2 of the analysis and only those pixels that 
were detected as having changed are classified in the 
Date 2 imagery. A traditional post-classification com- 
parison can then be applied using the Date | and Date 
2 classifications to yield more detailed “from-to” 
change information. 


This method may reduce change detection errors 
(omission and commission) and it provides detailed 
“from-to” change class information. The technique re- 
duces effort by allowing analysts to focus on the small 
amount of area that has changed between dates. In 
most regional projects, the amount of actual change 
over a I- to 5-year period is probably no greater than 
10% of the total area. The method is complex, requir- 
ing a number of steps, and the final outcome is depen- 
dent on the quality of the “change/no—change” binary 
mask used in the analysis. 


USGS scientists used a version of this technique to up- 
date the 2001 National Land Cover Database (NLCD) 
to 2006 using Landsat TM imagery (Xian et al., 2009). 
They used the 2001 NLCD data as the Date | classifi- 
cation map. They processed 2001 and 2006 Landsat 
imagery to identify pixels that changed between 2001 
and 2006. They then classified only the pixels that 
changed to create the new 2006 land-cover dataset. 
Pixels that did not change in the 2006 land-cover map 
retained the land cover present in the 2001 NLCD. 


Chi-Square Transformation Change 
Detection 


Ridd and Liu (1998) introduced a chi-square transfor- 
mation change detection algorithm. It will work on any 
type of imagery but let us for the moment apply it to 
six bands of Landsat TM data obtained on two dates. 
The chi-square transformation 1s: 


TL-l 
Yixet = (X-M) ED (X-M) 


(12.13) 
where Y,,;x¢j 18 the digital value of the pixel in the out- 
put change image, X is the vector of the difference of 
the six digital values between the two dates for each 
pixel, M is the vector of the mean residuals of each 
band for the entire image, 7’ is the transverse of the ma- 
trix, and y is the inverse covariance matrix of the six 
bands between the two dates. The usefulness of the 
transformation in this context rests on the fact that Y 
is distributed as a chi-square random variable with p 
degrees of freedom where p is the number of bands. 
Y = 0 represents a pixel of no change. The user creates 


CHAPTER 12. CHANGE DETECTION 545 


Cross-Correlation Change Detection 
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FIGURE 12-38 Diagram of cross-correlation change detec- 
tion (imagery courtesy of Richland County GIS Department). 


the output change image and highlights pixels with 
varying amounts of Y. 


Cross-Correlation Change Detection 


Cross-correlation change detection makes use of an ex- 
isting Date | digital land-cover map and a Date 2 un- 
classified multispectral dataset (Koeln and Bissonnette, 
2000). The Date 2 multispectral dataset does not need 
to be atmospherically corrected or converted to per- 
cent reflectance; the original brightness values are suf- 
ficient. Several passes through the datasets are required 
to implement cross-correlation change detection. First, 
every pixel in the Date | land-cover map associated 
with a particular class c (e.g., forest) out of m possible 
classes is located in the Date 2 multispectral dataset 
(Figure 12-38). The mean (K,,) and standard deviation 
(o .x) of all the brightness values in each band k in the 
Date 2 multispectral dataset associated with class c 
(e.g., forest) in the Date | land-cover map are comput- 
ed. Next, every pixel (BV;;,) in the Date 2 scene associ- 
ated with class c is compared with the mean (X,,) and 
divided by the standard deviation (o ,;). This value is 
summed and squared over all k bands. This is per- 
formed for each class. The result is a Z-score associat- 
ed with each pixel in the scene: 


= BY je Ha, ° 
Lije = » - 7 | 


(oy 
k=1 Ck 


(12.14) 
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where 


Zijc 18 the Z-score for a pixel at location i,j in the 
Date 2 multispectral dataset associated with a par- 
ticular class c found in the Date 1 land-cover map; 


cis the Date | land-cover class under investigation; 


nis the number of bands in the Date 2 multispectral 
image; 


k is the band number in the Date 2 multispectral 
image; 


BV jx is the brightness value of a pixel (or reflec- 
tance) at location ij in band k of the Date 2 multi- 
spectral dataset associated with a particular class 
found in the Date | land-cover map; 


Xx is the mean of all pixel brightness values found 
in band k of the Date 2 multispectral dataset associ- 
ated with a particular class c in the Date | land-cov- 
er map; 


Oo 1s the standard deviation of all pixel brightness 
values found in band k of the Date 2 multispectral 
dataset associated with a particular class c in the 
Date | land-cover map. 


The mean (X,,) and standard deviation (o ,;) values 
derived from the cross-correlation of a Date 1 four- 
class land-cover map with a Date 2 three-band multi- 
spectral dataset would be stored in a table like Table 
12-5 and used in Equation 12.14. 


The Z-statistic describes how close a pixel’s response is 
to the expected spectral response of its corresponding 
class value in the land-cover map (Civco et al., 2002). 
In the output file, the greater the Z-score of an individ- 
ual pixel, the greater the probability that its land cover 
has changed from Date | to Date 2. If desired, the im- 
age analyst can examine the Z-score image file and se- 
lect a threshold that can be used to identify all pixels in 
the scene that have changed from Date | to Date 2. 
This information can be used to prepare a “change/no- 
change” map of the area. 


Advantages associated with cross-correlation change 
detection include the fact that it is not necessary to 
perform an atmospheric correction on any dataset. It 
also eliminates the problems associated with phenolog- 
ical differences between dates of imagery. Unfortunate- 
ly, this change detection method is heavily dependent 
upon the accuracy of the Date 1 land-cover classifica- 
tion. If it has serious error, then the cross-correlation 
between Date | and Date 2 will contain error. Every 


TABLE 12-5 Hypothetical mean and standard deviations as- 
sociated with a cross-correlation change detection analysis 
of a Date 1 land-cover map with four classes and a Date 2 
multispectral image that contains three bands. 


Land-Cover 
Class Band 1 Band 2 Band 3 
1 M14 O11 M12 012 113 013 
2 H21 921 H22 022 H23 023 
3 H31 931 H32 032 133 033 
4 M41 | S41 | M42] S42 |] Haz | 943 


pixel in the Date | land-cover map must be assigned to 
a class. The method does not produce any “from-to” 
change detection information. 


Visual On-Screen Change Detection 
and Digitization 


A great amount of high-resolution remote sensor data 
is now available from satellite (e.g., IKONOS, GeoEye- 
1, WorldView-2, Pleiades) (Belward and Skoien, 2014). 
In addition, digital aerial photography is collected on 
an ongoing basis for many city, county, region, and 
state applications. Digital aerial photography is almost 
always collected rapidly after a major natural or hu- 
man-made disaster. It is not always necessary or practi- 
cal to use the aforementioned digital image processing 
change detection techniques. Rather, less complex 
techniques can be used. For example, it is possible to 
display two registered images on a computer screen 
and perform manual on-screen digitization to identify 
change polygons. This can be done using a digital im- 
age processing system or a GIS. 


Hurricane Hugo Example 


For example, consider the pre- and post-Hurricane 
Hugo black-and-white aerial photography shown in 
Figure 12-39ab. You would assume that it would be 
straightforward to use one of the change detection 
techniques (e.g., image differencing, change vector 
analysis) to identify the houses that were destroyed 
when Hurricane Hugo made landfall on September 21, 
1989. Unfortunately, due to the tremendous amount of 
sand and debris washed ashore during the hurricane, 
such digital techniques were of little value. The most 
useful method of identifying the important changes 
was to simply photointerpret the two images on the 
screen and then digitize the important information 
onto the post-Hurricane Hugo image (Figure 12-39c). 
The photointerpretation was performed using the fun- 
damental elements of image interpretation such as size, 
shape, shadow, tone, color, texture, site and association 
(Jensen, 2007). Hb mes that were completely destroyed 


CHAPTER 12. CHANGE DETECTION 547 


Visual On-Screen Change Detection of the Impact of Hurricane Hugo on Sullivan’s Island, SC 


A ON ATE Bee a PA 
A ey ¥ ; 


a. Pre-Hurricane Hugo orthophoto July 1, 1988. b. Post-Hurricane Hugo orthophoto October 5, 1989. 
ie ai 


c. Change information annotated on the October 5, 1989, orthophoto. 


FIGURE 12-39 a) Panchromatic orthophoto of Sullivan's Island, SC, obtained on July 1, 1988, prior to Hurricane Hugo. The 
data were rectified to State Plane Coordinates and resampled to 0.3 M 0.3 m spatial resolution. b) Panchromatic orthophoto 
obtained on October 5, 1989, after Hurricane Hugo. c) Change information overlaid on October 5, 1989, post-Hurricane 
Hugo aerial photograph, Sullivan's Island, SC. Completely destroyed houses are outlined in red. Partially destroyed houses 
are outlined in yellow. A red arrow indicates the direction of houses removed from their foundations. Three beachfront man- 
agement setback lines are shown in cyan (base, 20 year, 40 year). Areas of beach erosion are depicted as black lines. Areas of 
beach accretion caused by Hurricane Hugo are shown as dashed black lines. 
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are outlined in red, partially destroyed homes are out- 
lined in yellow, and the movement of homes off their 
foundations is shown using red lines with arrowheads 
(Figure 12-39c). 


Hurricane Katrina Example 


The same logic was applied to identify important 
change associated with Hurricane Katrina, which 
made landfall on August 29, 2005 (Figure 12-40a). 
Low oblique and vertical aerial photography of the ef- 
fects of Hurricane Katrina in Gulfport, MS, are shown 
in Figure 12-40bc. Hurricane Katrina storm surge did 
great damage to coastal geomorphology, vegetation, 
and especially human-made structures. A USGS high 
spatial resolution (1.2 M 1.2 ft.) color vertical aerial 
photograph of the Grand Casino in Gulfport, MS, is 
shown in Figure 12-40b. Note that approximately one- 
fourth of the Grand Casino was broken off by storm 
surge and moved onto Highway 90. Also note how nu- 
merous marine cargo containers are now located in- 
land. The low oblique aerial photography makes it 
easy to determine what change has taken place with 
the large section of the Grand Casino located in the 
center of Highway 90 (Figure 12-40c). Also note the 
debris piled up against the hotel in the background. 


Aral Sea Example 


Sometimes it is useful to simply visually examine mul- 
tiple dates of remotely sensed imagery to appreciate 
processes at work. For example, consider the change in 
the Aral Sea in Kazakhstan from 1973 to 2000. The 
shoreline for a portion of the Aral Sea recorded by the 
Landsat MSS in 1973 and 1987 and the Landsat 
ETM” in 2000 is shown in Figure 12-41. 


The Aral Sea is actually a lake, a body of fresh water. 
Unfortunately, more than 60 percent of the lake has 
disappeared in the last 30 years. Farmers and state of- 
fices in Uzbekistan, Kazakhstan, and Central Asian 
states began diverting river water to the lake in the 
1960s to irrigate cotton fields and rice paddies. In 1965, 
the Aral Sea received about 50 cubic kilometers of 
fresh water per year—a number that fell to zero by the 
early 1980s. Concentrations of salts and minerals be- 
gan to rise in the lake. The change in chemistry led to 
alterations in the lake’s ecology, causing precipitous 
drops in the fish population. The commercial fishing 
industry employed 60,000 people in the early 1960s. By 
1977, the fish harvest was reduced by 75 percent, and 
by the early 1980s the commercial fishing industry was 
gone. 


The shrinking Aral Sea has also had a noticeable effect 
on the region’s climate. The growing season is now 
shorter, causing many farmers to switch from cotton to 
rice, which requires even more diverted water. A sec- 


ondary effect of the reduction in the Aral Sea’s overall 
size is the rapid exposure of the lake bed. Strong winds 
that blow across this part of Asia routinely pick up and 
deposit tens of thousands of tons of now-exposed soil 
every year. This process has caused a reduction in air 
quality for nearby residents and affected crop yields 
due to the salt-laden particles falling on arable land. 


Environmental experts agree that the current situation 
cannot be sustained. Yet, driven by poverty and their 
dependence upon exports, officials in the region have 
failed to take any preventive action, and the Aral Sea 
continues to shrink (NASA Aral Sea, 2014). 


National Land Use/Cover Database of China 
Example 


Visual on-screen digitization change detection was re- 
cently used to identify change in the National Land 
Use/Cover Database of China at 1:100,000 scale using 
primarily Landsat TM imagery (Zhang et al.,2014). 


|_| nana Correction for 


Change Detection 


Now that many of the most widely adopted change de- 
tection algorithms have been identified, it is useful to 
provide some general guidelines about when it is neces- 
sary to atmospherically correct the individual dates of 
imagery used in the change detection process. 


When Atmospheric Correction Is 
Necessary 


Atmospheric correction of multiple-date remote sensor 
data is required when the individual date images used 
in the change detection algorithm are based on linear 
transformations of the data, e.g., a normalized differ- 
ence vegetation index image is produced for Date | and 
Date 2. The additive effects of the atmosphere on each 
date contaminate the NDVI values and the modifica- 
tion is not linear (Song et al., 2001). Contributions 
from the atmosphere to NDVI values are significant 
and can amount to 50% or more over thin or broken 
vegetation cover (McDonald et al., 1998; Song et al., 
2001). Similarly, the imagery should be atmospherical- 
ly corrected if the change detection is based on multi- 
ple-date red/near-infrared ratioed images (e.g., Landsat 
TM 4/TM 3). This suggests that it may be necessary to 
normalize or atmospherically correct the multiple-date 
imagery used to compute the linearly transformed data 
(e.g., NDVI) when the goal is to identify biophysical 
change characteristics through time rather than just 
land-cover change through time (Yuan and Elvidge, 
1998; Song et al., 2001; Du et al., 2002). 
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Visual On-Screen Change Detection of Hurricane Katrina Damage at Gulfport, MS 


a. GOES-12 August 29, 2005, visible band image c. Low-oblique photograph of a part of the 
overlaid on MODIS true-color background. Grand Casino moved by storm surge. 


Grand 
Casino 


b. Vertical aerial photograph of the Grand Casino and surrounding area obtained 
at a spatial resolution of 37 x 37 cm (1.2 x 1.2 ft). 


FIGURE 12-40 a) GOES-12 image of Hurricane Katrina making landfall on August 29, 2005, overlaid on MODIS true-color 
imagery (image courtesy of USGS Coastal and Marine Geology Program). b) Vertical high spatial resolution aerial photograph 
of the Grand Casino in Gulfport, MS, on August 29, 2005 (image courtesy of USGS Coastal and Marine Geology Program). 
c) Low oblique aerial photograph documenting the location of a part of the Grand Casino now lying on Highway 90 (photog- 
raphy courtesy of NOAA Remote Sensing Division). 
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Drawdown of the Aral Sea, Kazakhstan, from 1973 to 2000 


* i 


a. Landsat MSS image obtained on May 29, 1973. 


a 7 at 


PAs 


b. Landsat MSS image obtained on August 19, 1987. 


c. Landsat 7 ETMt image obtained on July 29, 2000. 


A portion of the Aral Sea shoreline recorded by the Landsat Multispectral Scanner (MSS) and Landsat 
Enhanced Thematic Mapper Plus (ETM*) in 1973, 1987, and 2000 (imagery courtesy of NASA Earth Observatory). 


A change/no-change map produced using image differ- 
encing logic and atmospherically corrected remote sen- 
sor data normally looks different from a change/no- 
change map produced using image differencing logic 
and non-atmospherically corrected data if the thresh- 
old boundaries are held constant in the change image 
histograms. However, if the analyst selects the appro- 
priate thresholds in the two tails of the change detec- 
tion image histogram, it doesn’t really matter whether 
the change detection map was produced using atmo- 
spherically corrected or non-atmospherically corrected 
data. But, if the analyst desires that all stable classes in 
the change image have a value of 0 in the change histo- 
gram then it is useful to normalize one image to anoth- 
er or atmospherically correct both images prior to 
performing image differencing. 


Obtaining quality training data is very expensive and 
time-consuming because it usually involves people and 
fieldwork. Therefore, it will become increasingly im- 
portant to be able to extend training data through both 
time and space. In other words, training data extracted 
from a Date | image should be able to be extended to a 
Date 2 image of the same geographic area (signature 
extension through time) or perhaps even to a Date | or 
Date 2 image of a neighboring geographic area (signa- 
ture extension through space). Extending training data 
through space and time will require that each image 
evaluated be atmospherically corrected to surface re- 
flectance whenever possible using one of the tech- 
niques described in Chapter 6. Nevertheless, it is not 
always necessary to correct remote sensor data when 
classifying individual dates of imagery or performing 
change detection. 


When Atmospheric Correction Is 
Unnecessary 


A number of studies have documented that it is unnec- 
essary to correct for atmospheric effects prior to image 
classification if the spectral signatures characterizing 
the desired classes are derived from the image to be 
classified. This is because atmospherically correcting a 
single date of imagery is often equivalent to subtract- 
ing a constant from all pixels in a spectral band. This 
action simply translates the origins in multidimension- 
al feature space. The class means may change, but the 
variance—covariance matrix remains the same irrespec- 
tive of atmospheric correction. In other words, atmo- 
spheric correction is unnecessary as long as the 
training data and the data to be classified are in the 
same relative scale (corrected or uncorrected) (Song et 
al., 2001). This suggests that it is not necessary to at- 
mospherically correct Landsat TM data obtained on 
Date 1 if it is going to be subjected to a maximum like- 
lihood classification algorithm and all the training data 
are derived from the Date 1 imagery. The same holds 
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true when a Date 2 image is classified using training 
data extracted from the Date 2 image. Change between 
the Date 1 and Date 2 classification maps derived from 
the individual dates of imagery (corrected or uncor- 
rected) can easily be compared in a post-classification 
comparison. 


Atmospheric correction is also unnecessary when 
change detection is based on classification of multiple- 
date composite imagery in which the multiple dates of 
remotely sensed images are rectified and placed in a 
single dataset and then classified as if it were a single 
image (e.g., multiple-date principal components 
change detection). Only when training data from one 
time and/or place are applied in another time and/or 
place is atmospheric correction necessary for image 
classification and many change detection algorithms. 
The use of relative radiometric correction to normalize 
intensities of bands of multitemporal imagery to a ref- 
erence scene is sufficient for many change detection ap- 
plications (Song et al., 2001; Coppin et al., 2004; Chen 
et al., 2012). 


ser 


A one-time inventory of natural resources is often of 
limited value. A time series of images and the detection 
of change provides significant information on the re- 
sources at risk and may be used in certain instances to 
identify the agents of change. Change information is be- 
coming increasingly important in local, regional, and 
global environmental monitoring (Woodcock et al., 
2001; Olofsson et al., 2014). This chapter identifies the 
remote sensor system and environmental variables that 
should be considered whenever a remote sensing 
change detection project is initiated. Several useful 
change detection algorithms were reviewed. Scientists 
are encouraged to carefully review and understand 
these principles so that accurate change detection can 
take place. 
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Lon 


Information derived from remotely sensed data is be- 
coming increasingly important for planning, ecosystem 
monitoring, food security and human health assess- 
ment, and police/military reconnaissance at local, re- 
gional, and global scales (NRC, 2007, 2009, 2012). The 
remote sensing—derived thematic information may be 
in the form of thematic maps or statistics derived from 
sampling techniques. The thematic information must 
be accurate because important decisions are made 
throughout the world using the geospatial information 
(Jensen and Jensen, 2013; Wickham et al., 2013). 


Unfortunately, remote sensing—derived thematic infor- 
mation contains error. Analysts who create remote 
sensing—derived thematic information should recog- 
nize the sources of the error, minimize it as much as 
possible, and inform the user how much confidence he 
or she should have in the thematic information. Re- 
mote sensing—derived thematic maps should normally 
be subjected to a thorough accuracy assessment before 
being used in subsequent scientific investigations and 
policy decision-making (Borengasser et al., 2008; Con- 
galton and Green, 2009). This also includes assessing 
the accuracy of remote sensing—derived change detec- 
tion maps (Foody, 2010; Burnicki, 2011). 


2 Overview 


This chapter first introduces the steps generally taken 
to assess the accuracy of remote sensing—derived the- 
matic map information (Figure 13-1). It is not con- 
cerned with the spatial (x, y) accuracy of remote 
sensing—derived information, which was discussed in 
Chapter 7. Sources of error in remote sensing—derived 


REMOTE SENSING—DERIVED THEMATIC 
MAP ACCURACY ASSESSMENT 


products are described. Error matrices (1.e., contingen- 
cy tables) are introduced. The difference between train- 
ing and test samples is discussed. Sample size and 
design are reviewed. Important information about col- 
lecting ground reference information at locations using 
a response design are presented. Various methods of 
analyzing the error matrix are discussed including 
qualitative evaluation and quantitative assessment. 
Fuzzy accuracy assessment is introduced. A method to 
quantify the accuracy of binary change detection maps 
is presented. Geostatistical analysis to assess the accu- 
racy of remote sensing—derived information is re- 
viewed. Finally, image metadata and_ lineage 
information that should be provided for remote sens- 
ing—derived products is discussed. 


Steps to Perform Accuracy 
Assessment 


The steps generally taken to assess the accuracy of the- 
matic information derived from remotely sensed data 
are summarized in Figure 13-1. First, it is important to 
clearly state the nature of the thematic accuracy assess- 
ment problem, including: 


¢ what the accuracy assessment process is expected to 
accomplish, 


* the classes of interest (discrete or continuous), and 


¢ the sampling design and sampling frame (consisting 
of area and list frames). 


The objectives of the accuracy assessment should be 
clearly identified. Sometimes a simple qualitative ex- 
amination may be appropriate if the remote sensing— 
derived data are to be used as general undocumented 


558 INTRODUCTORY DIGITAL IMAGE PROCESSING 


information. Conversely, if the remote sensing—derived 
thematic information will impact the lives of human 
beings, flora, fauna, etc., then it may be necessary to 
conduct a thorough probability design-based accuracy 
assessment. 


The accuracy assessment sampling design is the proto- 
col (i.e., instructions) by which the reference sample 
units are selected (Stehman and Czaplewski, 1998; 
Stehman, 2009). Sampling designs generally require a 
sampling frame that consists of the materials or devices 
which delimit, identify, and allow access to the ele- 
ments of the target population. There are two impor- 
tant elements associated with the sampling frame: 


¢ the area frame, and 
¢ the list frame. 


In a remote sensing context, the area frame represents 
the exact geographic dimension of the entire study ar- 
ea. The list frame is simply a list of all the possible sam- 
pling units within the area frame. The actual sample is 
selected solely from this list of sampling units. 


The accuracy assessment process is dependent upon 
the selection of a sampling unit (Stehman and 
Czaplewski, 1998). When conducting remote sensing— 
related accuracy assessment, analysts typically select 
one of three types of areal (spatial) sampling units: in- 
dividual pixels (not recommended by Congalton and 
Green, 2009), a block of pixels (recommended), or 
polygons (Stehman and Wickham, 2011). There is no 
consensus on the ideal type of sampling unit. If a pixel 
is selected as the sampling unit, the land cover associat- 
ed with it may be the result of a hard (crisp) classifica- 
tion in which each pixel is labelled as exactly one class, 
or a soft (fuzzy) classification in which each pixel has a 
membership value for each class (Stehman, 2009). 


Once the accuracy assessment problem has been stat- 
ed, then there are two analysis methods that may be 
used to validate the accuracy (or assess the error) of a 
remote sensing—derived thematic map: 


* qualitative confidence-building assessment, and 
* statistical analysis. 


A confidence-building assessment involves the visual ex- 
amination of the thematic map associated with the 
overall area frame by knowledgeable individuals to 
identify any gross errors (sometimes referred to as 
blunders) such as urban areas in water bodies or unre- 
alistic classes on mountain tops. If major errors are 
identified and we have no qualitative confidence in the 
map, then the thematic information should be discard- 
ed and the classification process iterated using more 
appropriate logic. If the thematic map appears to be 
reasonable, then the analyst may proceed to statistical 


confidence-building measurements (Morisette et al., 
2005). 


Statistical analysis may be subdivided into two types 
(Stehman, 2000, 2001): 


¢ model-based inference, and 
¢ design-based inference. 


Model-based inference is not concerned with the accu- 
racy of the thematic map. It is concerned with estimat- 
ing the error of the remote sensing classification 
process (or model) that actually generated the thematic 
map. Model-based inference can provide the user with 
a quantitative assessment of each classification deci- 
sion. For example, each pixel in a MODIS-derived 
land-cover product has a confidence value that mea- 
sures how well the pixel fits the training examples pre- 
sented to the classifier (Morisette et al., 2005). 


Design-based inference is based on statistical principles 
that infer the statistical characteristics of a finite popu- 
lation based on the sampling frame. Some common 
statistical measurements include producer’s error, con- 
sumer’s error, overall accuracy, and Kappa coefficient 
of agreement (Congalton and Green, 2009; Stehman, 
2009). Design-based inference is expensive yet power- 
ful because it provides unbiased map accuracy statis- 
tics using consistent estimators. 


Before we describe how to extract the unbiased statisti- 
cal measurements, however, it is useful to briefly review 
the sources of error that may be introduced into a re- 
mote sensing—derived product (e.g., a land-use or land- 
cover map) (Figure 13-2). This will help us appreciate 
why an accuracy assessment is so important when the 
remote sensing—derived information is used to make 
decisions. 


Sources of Error in Remote 
Sensing—Derived Thematic 
Maps 


First, error may be introduced during the remote sens- 
ing data acquisition process (Figure 13-2). It is quite 
common for remote sensing system detectors, cameras, 
RADARs, LiDARs, etc. to become miscalibrated. This 
can result in inaccurate remote sensing measurements 
(e.g., multispectral radiance, RADAR backscatter, Li- 
DAR laser intensity). The aircraft or spacecraft plat- 
form may roll, pitch, and/or yaw during data 
acquisition. Erroneous commands may be entered into 
the data-collection or inertial navigation system from 
ground control personnel. Finally, the scene being im- 
aged may be randomly affected by unwanted haze, 
smog, fog, dust, high humidity, or sunglint that can 
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General Steps Used to Assess the Accuracy of Thematic 
Information Derived from Remotely Sensed Data 


State the nature of the thematic accuracy assessment problem. 
* State what the accuracy assessment is expected to accomplish. 
* Identify classes of interest (discrete or continuous). 
* Specify the sampling frame within the sampling design: 
- area frame (the geographic region of interest) 
- list frame (consisting of points or areal sampling units) 
Select method(s) of thematic accuracy assessment. 


* Confidence-building assessment: 


- Qualitative 
* Statistical measurement: 


- Model-based inference (concerned with image processing methods) 
- Design-based statistical inference of thematic information 
Compute total number of observations required in the sample. 


* Observations per class. 
Select sampling design (scheme). 

* Random. 

* Systematic. 

* Stratified random. 


* Stratified systematic unaligned sample. 


* Cluster sampling. 


Obtain ground reference data at observation locations using a response design. 


* Evaluation protocol. 
* Labeling protocol. 

Error matrix creation and analysis. 
* Creation: 


- Ground reference test information (columns) 
- Remote sensing classification (rows) 


* Univariate statistical analysis: 
- Producer’s accuracy 
- User’s accuracy 
- Overall accuracy 


* Multivariate statistical analysis: 


- Kappa coefficient of agreement; conditional Kappa 


- Fuzzy 


Accept or reject previously stated hypothesis. 


Distribute results if accuracy is acceptable. 


* Accuracy assessment report. 
* Digital products. 
* Analog (hard-copy) products. 


* Image and map lineage report. 


FIGURE 13-1 The general steps used to assess the accuracy of remote sensing—derived thematic information. 


dramatically affect the quality and accuracy of the in- 
formation that is extracted. 


Considerable effort is spent preprocessing the remotely 
sensed data to remove both the geometric and radio- 
metric error inherent in the original imagery (see 
Chapters 6, 7, and 12). Unfortunately, even after pre- 
processing, there is always residual geometric and ra- 
diometric error. For example, residual geometric error 
may cause pixels still to be in incorrect geographic lo- 
cations. Similarly, even the best atmospheric correction 
will not result in a perfect relationship between percent 
reflectance measured on the ground and percent reflec- 
tance measured by an optical remote sensing system 


for the same geographic area. The residual geometric 
and radiometric error present in the imagery influences 
subsequent digital image processing (Lunetta et al., 
1991; Lunetta and Lyon, 2005; Jensen et al., 2009). 


Sometimes qualitative or quantitative information ex- 
traction techniques are based on flawed logic. For ex- 
ample, a hard land-cover classification scheme may not 
contain classes that are mutually exclusive, exhaustive, 
and hierarchical. Training sites may be incorrectly la- 
beled during the training phase of a supervised classifi- 
cation. Clusters may be incorrectly labeled when 
performing an unsupervised classification (Chapter 9). 
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Sources of Error in Remote Sensing—Derived Information 


FIGURE 13-2 Remote sensing—derived products such as land-use and land-cover maps contain error. The error accumulates as 
the remote sensing data are collected and various types of processing take place. An error assessment is necessary to identify 
the type and amount of error in a remote sensing-derived product (based on Figure 1 in Lunetta, R. S., Congalton, R. G., 
Fenstermaker, L. K., Jensen, J. R., McGwire, K. C., and L. R. Tinney, 1991, “Remote Sensing and Geographic Information System 
Data Integration: Error Sources and Research Issues,” Photogrammetric Engineering & Remote Sensing, 57(6):677-687). 


A human interpreter performing visual image interpre- 
tation may label an image-derived polygon incorrectly. 


Thematic information derived from the remotely 
sensed data may be in raster (grid) or polygon carto- 
graphic data structures. Sometimes it is desirable to 
convert the data from one data structure to another, 
e.g., raster-to-vector or vector-to-raster. The conver- 
sion process may introduce error. 


All the error discussed so far accumulates in the re- 
mote sensing—derived information (e.g., a land-use or 
land-cover map). Therefore, it is essential to conduct 
an error assessment (or accuracy assessment) to place 
confidence limits on the remote sensing—derived infor- 
mation. Unfortunately, the person conducting the er- 
ror evaluation could make critical errors regarding the 
ground reference data sampling design, the number of 
samples obtained per class, the geometric accuracy of 
the sample data, and data autocorrelation. In addition, 
the person(s) collecting ground reference information 
in the field often make mistakes (e.g., mislabeling). 
Therefore, it is imperative to use the term ground refer- 


ence information rather than ground-truth because al- 
most all ground reference information will contain 
some error (Stehman and Wickham, 2011). 


Hopefully, a robust multivariate statistical analysis is 
used so that confidence limits can be specified for the 
remote sensing—derived thematic information (e.g., 
hectares of agriculture, forestry). 


The final remote sensing—derived product is created 
when the thematic information is judged to be suffi- 
ciently accurate. Unfortunately, the analyst can still in- 
troduce error by not following standard cartographic 
procedures for the creation of scale bars and map leg- 
ends. Metadata may also contain error. 


Finally, the people who actually make the decisions of- 
ten do not understand or appreciate the amount of er- 
ror that has accumulated in the remote sensing—derived 
thematic product (Meyer and Werth, 1990). They 
should be made aware of the amount of error in the 
product and be cautioned not to overstate the level of 
accuracy. 
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TABLE 13-1 Sample-based error matrix where there are k classes in the remote sensing—derived thematic map and N ground 


reference test samples that are used to assess its accuracy. 


Row total 

Xi+ 

x11 X42 X13 oe Xk X44 

X21 X22 X23 rare X2k X24 

BA 2G 2383 aoo6 2S X34 

Xk Xk,2 Xk3 babe Xk,k Xk 

Column X44 X42 X43 6 de Xek N 
total 
Xj 

where: 


e All off diagonal cells represent misclassified pixels. 


* Cell entry, xj, is the proportion of area mapped as class i and labeled class j in the reference data. 
° The row marginal, x;,, is the sum of all xj values in row i and represents the proportion of area classified as class i. 
* The column marginal, x,;, is the sum of all xj values in column j and represents the proportion of area that is truly class j. 


e The diagonal, x;;, summarizes correctly classified pixels. 


Classification accuracy (or error) assessment was an 
afterthought rather than an integral part of many re- 
mote sensing studies in the 1970s and 1980s. In fact, 
many studies still simply report a single number (e.g., 
78%) to express classification accuracy. Such non-site- 
specific accuracy assessments completely ignore loca- 
tional accuracy. In other words, only the total amount 
of a category is considered without regard for its loca- 
tion. A non-site-specific accuracy assessment generally 
yields high accuracy but misleading results when all the 
errors balance out in a region. 


The Error Matrix 


To determine the accuracy of a remote sensing—derived 
thematic map it is necessary to systematically compare 
two sources of information: 


1. pixels or polygons in the remote sensing—derived 
classification map, and 

2. ground reference test information obtained at the 
same x, y location (which may in fact contain 
error). 


The relationship between these two sets of information 
is commonly summarized in an error matrix (some- 
times referred to as a contingency table or confusion 


matrix). The error matrix provides the basis on which 
to both describe classification accuracy and character- 
ize errors, which may help refine the classification or 
the estimates derived from it (Congalton and Green, 
2009; Stehman, 2009). 


Stehman and Foody (2009) summarize the characteris- 
tics of a population-based error matrix where the ana- 
lyst has reference information about the entire 
population within the study area (e.g., a wall-to-wall 
land cover reference map that can be compared to the 
wall-to-wall remote sensing—derived thematic map). 
This is a relatively rare occurrence. Most of the time it 
is necessary to collect samples within the study area at 
specific x, y locations that can be compared with those 
same x, y locations in the remote sensing—derived the- 
matic map. The characteristics of a sample-based error 
matrix are shown in Table 13-1. 


The error matrix is used to assess the remote sensing 
classification accuracy of the k classes in the remote 
sensing—derived thematic map. The central part of the 
error matrix is a square array of numbers & x k in size 
(e.g., 3 X 3). The columns of the matrix represent the 
ground reference test information and the rows corre- 
spond to the classification generated from analysis of 
the remotely sensed data. The intersection of the rows 
and columns summarizes the number of sample units 
(i.e., pixels, clusters of pixels, or polygons) assigned to 
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a particular category (class) relative to the actual cate- 
gory as verified in the field. The total number of sam- 
ples examined is N. 


The diagonal of the matrix (e.g., x),;, x2.) summarizes 
those pixels or polygons that were assigned to the cor- 
rect class. Every error in the remote sensing classifica- 
tion relative to the ground reference information is 
summarized in the off-diagonal cells of the matrix (e.g., 
X 1.25 X2,1, 2,3). Each error is both an omission from the 
correct category and a commission to a wrong catego- 
ry. The column and row totals around the margin of 
the matrix (referred to as marginals) are used to com- 
pute errors of inclusion (commission errors) and errors 
of exclusion (omission errors). The outer row and col- 
umn totals are used to compute producer’s and user’s 
accuracy (to be discussed). Some recommend that the 
error matrix contain proportions rather than individu- 
al counts (Stehman and Czaplewski, 1998). 


But how do we a) obtain unbiased ground reference test 
information to compare with the remote sensing classi- 
fication map and fill the error matrix with values, and 
b) perform the error (or accuracy) assessment? Basical- 
ly, we must: 


¢ be aware of the problems associated with using 
training versus ground reference test information; 


¢ determine the total number of samples to be col- 
lected for each thematic category; 


¢ design an appropriate sampling scheme (using tradi- 
tional and/or geostatistical techniques); 


¢ obtain ground reference information at sample loca- 
tions using a response design; and 

* apply appropriate descriptive and multivariate sta- 
tistics (normal and/or fuzzy) to assess the accuracy 
of the remote sensing—derived information. 


ze Training versus Ground 
Reference Test Information 


Some image analysts perform an accuracy assessment 
based only on the training pixels (or training polygons 
if the study is based on human visual interpretation or 
object-based image analysis—OBIA) used to train 
classification algorithm. Unfortunately, the locations 
of these training sites are usually not random. They are 
biased by the analyst’s a priori knowledge of where cer- 
tain land-cover types exist in the scene. Because of this 
bias, an accuracy assessment based on how well the 
training class data were classified usually results in 
higher classification accuracies than an accuracy as- 
sessment based on unbiased ground reference test in- 
formation (e.g., Muchoney and Strahler, 2002). 


The ideal situation is to locate ground reference test 
pixels (or polygons) in the study area. These sites are 
not used to train the classification algorithm and there- 
fore represent unbiased reference information. It is 
possible to collect some ground reference test informa- 
tion prior to the classification, perhaps at the same 
time as the training data. But the majority of test refer- 
ence information is often collected after the classifica- 
tion has been performed using a random sample to 
collect the appropriate number of unbiased observa- 
tions per category (to be discussed). 


Landscapes often change rapidly. Therefore, it is best 
to collect both the training and ground reference test 
information as close to the date of remote sensing data 
acquisition as possible. This is especially important 
where the land use or land cover changes rapidly with- 
in a single season (Foody, 2010). For example, agricul- 
ture and illicit drug crops often have relatively short 
growing seasons or are harvested rapidly and then re- 
peatedly during a season. If a scientist waits too long 
to obtain the ground reference test information, it may 
not be possible to tell what was actually in the field the 
day of the remote sensing data acquisition. This is a se- 
rious condition because it becomes very difficult to 
populate an error matrix with accurate information. 


Ideally, the ground reference test data are obtained by 
visiting the site on the ground and making very careful 
observations that can then be compared with the re- 
mote sensing—derived information for the exact loca- 
tion (McCoy, 2005). Unfortunately, sometimes it is 
difficult to actually visit all the sites identified in the 
random sample. Some sites selected in the random 
sample may be completely inaccessible due to extreme- 
ly rugged terrain. Others may be inaccessible because 
private land owners, government agencies, or even 
criminals (e.g., illegal drug cultivators) deny access. 


When this occurs, scientists sometimes obtain higher- 
spatial-resolution remotely sensed data, interpret it, 
and use it as a surrogate for ground reference test infor- 
mation (e.g., Morisette et al., 2005). In such cases, the 
general rule of thumb is that the imagery used to ob- 
tain the ground reference test information should be 
substantially higher in spatial and/or spectral resolu- 
tion than the imagery used to derive the original the- 
matic information. For example, many studies have 
used high spatial resolution aerial photography (e.g., 
nominal spatial resolution <1 x | m) to extract ground 
reference test information that is then compared with 
thematic data produced using Landsat Thematic Map- 
per imagery (30 x 30 m). Admittedly, this is not the 
best solution. However, sometimes it is the only alter- 
native if one wants to populate an error matrix and 
perform an accuracy (error) assessment. 
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At some point we must compare the results of the 
ground reference test information with the remote 
sensing—derived information at the appropriate loca- 
tion in the thematic map. When we locate a ground ref- 
erence test unit in the remote sensing—derived thematic 
map at pixel 7, 7 we do not usually just compare the pix- 
el at that location with the ground reference test data. 
Rather, we often look at the 8 pixels surrounding pixel 
i,j and perhaps label the pixel according to the class 
that has the highest frequency of occurrence (e.g., if 
there are 3 corn and 5 soybean pixels in the 3 x 3 win- 
dow surrounding the pixel at location i,7, then we 
would label the pixel soybean for error matrix evalua- 
tion purposes). This procedure is common practice 
and minimizes the effects of geometric misregistration 
in the remote sensing—derived product (Jensen et al., 
1993). 


tm Sample Size 


The number of ground reference test samples to be 
used to determine the accuracy of individual categories 
in a remote sensing classification map is an important 
consideration (Foody, 2009). Usually, a design-based 
inferential framework is adopted in accuracy assess- 
ment programs (Stehman, 2000; 2009). With such a 
framework, an appropriately sized sample may be esti- 
mated using conventional statistics (Foody, 2002). 


Some analysts use an equation based on the binomial 
distribution or the normal approximation to the bino- 
mial distribution to compute the required sample size. 
Others suggest that a multinomial distribution be used 
to determine the sample size because we are usually in- 
vestigating the accuracy of multiple classes of informa- 
tion on a land-cover map (Congalton and Green, 
2009). Alternatively, a model-based inferential frame- 
work may be adopted based on geostatistical analysis 
(which takes into account autocorrelation) to design 
an efficient sample (Stehman, 2000; Foody, 2002). 


Sample Size Based on Binomial 
Probability Theory 


Fitzpatrick-Lins (1981) suggested that the sample size 
N to be used to assess the accuracy of a land-use classi- 
fication map be determined from the formula for the 
binomial probability theory: 


y = 200M) 
E 


where p is the expected percent accuracy of the entire 
map, g = 100 — p, E is the allowable error, and Z = 2 
from the standard normal deviate of 1.96 for the 95% 


(13.1) 


two-sided confidence level. For a sample for which the 
expected accuracy is 85% at an allowable error of 5%, 
the number of points necessary for reliable results is 
(McCoy, 2005): 


2 
N= 2 ES) = a minimum of 204 points. 
5 


With expected map accuracies of 85% and an accept- 
able error of 10%, the sample size for a map would be 
51: 


_ 2°(85)(15) _ 


N 
107 


Ss 


Thus, the lower the expected percent accuracy (p), and 
the greater the allowable error (£), the fewer ground 
reference test samples that need to be collected to eval- 
uate the classification accuracy. 


Early remote sensing research placed emphasis on 
achieving remote sensing-derived thematic map prod- 
ucts that were approximately 85% (Anderson et al., 
1976). Foody (2008) points out that the widely used 
target accuracy of 85% can be pessimistically biased 
and too restrictive when compared to other fields of 
endeavor. Pontius and Millones (2011) suggest that it is 
not necessary to be concerned with a universal stan- 
dard for accuracy because it is not related to any spe- 
cific research question or study area. 


Sample Size Based on Multinomial 
Distribution 


Analysts usually make thematic maps that contain 
multiple classes (e.g., vegetation, bare soil, water, ur- 
ban), not just binary information (e.g., urban versus 
everything else). Therefore, some scientists prefer to 
use equations based on a multinomial distribution to 
determine the sample size necessary to assess classifica- 
tion accuracy. Sample size (\V) derived from a multino- 
mial distribution is based on the equation (Congalton 
and Green, 2009): 


y = BUH) 
2 
i 


(13.2) 
b 


where IT; is the proportion of a population in the ith 
class out of k classes that has the proportion closest to 
50%, b; is the desired precision (e.g., 5%) for this class, 
B is the upper (a/k) X 100th percentile of the chi 
square (y ~) distribution with | degree of freedom, and 
k is the number of classes. 
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For example, suppose a land-cover map contains eight 
classes (k = 8) and we know that class IT, occupies ap- 
proximately 30% of the map area and that this propor- 
tion is closest to 50%. Furthermore, we desire a level of 
confidence of 95% and a precision (b;) of 5%. B is de- 
termined from the y7 table with 1 degree of freedom 
and l-a/k as x (1,0,99375) = 7.568 (Congalton and 
Green, 2009): 


0.05 _ 


1 0.99375 


_ 7.568(0.30)(1 — 0.30) 


N 
0.057 


— 1.58928 _ 
= 9.0025 636 samples 


Therefore, in this example 636 samples should be ran- 
domly selected to adequately fill the error matrix. Ap- 
proximately 80 samples per class are required (e.g., 80 
x 80 = 640). 


If we have no idea about the proportion of any of the 
classes in the land-cover map, then we can use the 
worst-case multinomial distribution algorithm where 
we assume that one class occupies 50% of the study 
area (Congalton and Green, 2009): 
n=4%. 


4b* 


Holding the precision constant at 5% for all k classes 
yields: 


(13.3) 


N= ec 757 samples . 


4(0.057) 


Thus, 757 random samples would have to be obtained 
because we did not have prior knowledge about the 
true proportion of any of the k classes in this worst- 
case scenario. 


Sometimes a confidence interval of 95% is unrealistic. 
For example, if we relax the confidence interval from 
95% to 85%, as is standard for many land-use and 
land-cover mapping products as previously discussed, 
the new y *(1,0.98125) value for B = 5.695. If we main- 
tain the same desired precision (1.e., 5%), then the total 
number of samples required drops to 478: 


a, 0.15 _ 
i 1 8 0.98125 


1 


_ 5,695(0.30)(1 — 0.30) 
0.05° 


_— 1.19595 _ 
= 9.0025> 478 samples . 


N 


Therefore, approximately 60 samples are required per 
class (e.g., 8 x 60 = 480). The worst-case scenario 
would require a total of 570 samples (Congalton and 
Green, 2009): 


5.695 


a 2 
4(0.057) 


= 570 samples 


or approximately 71 samples per class (8 X 71 = 568). 


Unfortunately, it is not always possible to obtain such 
large numbers of random samples. A balance between 
what is statistically sound and what is practicably at- 
tainable must be found. Congalton (1991) and Congal- 
ton and Green (2009) suggest that a good rule of 
thumb is to collect a minimum of 50 samples for each 
land-cover class in the error matrix. Foody (2009) says 
this approach has a number of attractions, not least its 
simplicity and in the provision of a sample that allows 
accuracy to be assessed on a per-class basis. If the area 
is especially large (e.g., >1 million ha) or the classifica- 
tion has a large number of land-use categories (e.g., 
>10 classes), the minimum number of samples should 
be increased to 75 or 100 samples per class (McCoy, 
2005). The number of samples can also be adjusted 
based on the relative importance of that category with- 
in the objectives of the project or by the inherent vari- 
ability within each category. It may be useful to take 
fewer samples in categories that show little variability, 
such as water and forest, and increase the sampling in 
the categories that are more variable, such as agricul- 
tural areas. The goal is to obtain an unbiased, repre- 
sentative sample that can be used to populate an error 
matrix. 


2 Sampling Design (Scheme) 


Once the total sample size (V ) and the number of sam- 
ples required per class (strata) are determined, it is nec- 
essary to determine the geographic location (x, y) of 
these N samples in the real world so that we can visit 
them and obtain ground reference test information. 
The sample locations must be determined randomly 
without bias. Any bias will cause the statistical analysis 
of the error matrix to over- or underestimate the true 
accuracy of the thematic map. Therefore, it is necessary 
to select an appropriate random sampling design 
(scheme). Each sampling scheme assumes a different 
sampling model and consequently different variance 
equations. 


There are numerous sampling designs that can be used 
to collect ground reference test data for assessing the 
accuracy of a remote sensing—derived thematic map 
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Sampling Methods 


a. Random sampling. 


Irregularly Spaced y 


Irregularly Spaced x 


c. Stratified random sampling. 


Stratum | |@ 


Stratum 4 
forest 


(McCoy, 2005; Congalton and Green, 2009; Stehman, 
2009). Some of the most commonly used include: 


* random sampling, 

* systematic sampling, 

* stratified random sampling, 

* stratified systematic unaligned sampling, and 
* cluster sampling. 


Simple Random Sampling 


The key to accurately projecting the characteristics of a 
sample onto a population is to draw the sample with 
care to ensure that it is truly representative of its popu- 
lation (Jensen and Shumway, 2010). Simple random 
sampling without replacement generally provides ade- 
quate estimates of the population parameters, provided 


b. Systematic sampling. 
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FIGURE 13-3 Several geographic sampling methods 
may be used to infer the statistical characteristics of a 
population: a) Simple random sampling. b) Systematic 
sampling that is initiated with random x,y coordinates 
selected for point A. A biased sample would likely be 
extracted if the systematic sample were drawn from 
terrain with ridge and valley topography orientation 
such as that shown. c) Stratified random sampling. This 
particular study area has been subdivided into four 
strata. d) Stratified systematic unaligned sampling. 

e) Cluster sampling. 


the sample size is sufficient (Figure 13-3a) (Congalton, 
1988). Normally a random number generator is used 
to identify random x, y coordinates within the study 
area. Samples located randomly throughout a hypo- 
thetical study area are shown in Figure 13-3a. The 
method ensures that the members of a population are 
drawn independently with equal probabilities of being 
selected and that the selection of one sample does not 
influence the selection of any other sample. An advan- 
tage of working with random samples is that an impor- 
tant assumption underlying many statistical techniques 
is satisfied. 


Unfortunately, random sampling may undersample 
small but possibly very important classes (e.g., the 
small acreage of illicit crops in a large agricultural re- 
gion) unless the sample size is significantly large (van 
Oort, 2007). The random samples may also be located 
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Sampling Alternatives 40 m 


c. Stratified random sample. 


FIGURE 13-4 Hypothetical remote sensing—derived the- 
matic map consisting of 100 pixels with 75% classified as 
corn and 25% classified as soybean. a) A random distribu- 
tion of 12 points allocated throughout the study area. Un- 
fortunately, only two points fall within the soybean field 
which should ideally be allocated three samples. Soybeans 
are under-sampled. b) Systematic sampling of the study 
area every 40 m in the x-and y-directions. Unfortunately, 
only one sample is allocated within the soybean field. It is 
undersampled. c) The random allocation of 12 samples us- 
ing stratified-random sampling. As expected, 75% of the 
samples (9) are allocated to corn and 25% of the samples (3) 
are allocated to soybeans. Both received the requisite num- 
ber of samples relative to the proportion of area they occu- 
py in the classification map. 


in inhospitable or access-denied locations. Finally, 
some clustering of the random sample points often re- 
sults and, as a consequence, important spatial proper- 
ties of the population may be overlooked. 


A practical example of random sampling is shown in 
Figure 13-4a. A remote sensing—derived thematic map 
consisting of 100 pixels with 75% classified as corn and 
25% classified as soybean is shown. Due to cost con- 
straints, only 12 samples could be visited in the field for 
this particular study area. The classification of the re- 
mote sensing—derived thematic map at each sample lo- 
cation would be compared with the land cover found in 
the field at the same location. Ten of the 12 randomly 
located samples fall on corn (83%). Two samples fall 
on soybean. It is clear that soybeans are undersampled 


(17%) relative to the proportion of area that they occu- 
py (25%) in the remote sensing—derived thematic map. 
Interestingly, collecting more random samples (e.g., 
100) does not guarantee that the soybean area will ever 
receive any more samples. 


Systematic Sampling 


To avoid the shortcomings of random sampling re- 
garding the coverage of an area, a systematic sampling 
plan may be used (Jensen and Shumway, 2010). This 
procedure requires that it be possible to work through 
the study area in some consistent and orderly manner. 
The sampling is normally initiated by the random se- 
lection of the starting point x, y coordinates (A) as 
shown in Figure 13-3b. Unfortunately, if there are peri- 
odicities in the data being collected (e.g., the ridge and 
valley topography of the Appalachian Mountains), the 
regularly spaced points could hit the same point on a 
cycle (e.g., a valley) time and again, yielding biased 
sample estimates of the population. Systematic sam- 
pling should be used with caution because it may over- 
estimate the population parameters (Congalton, 1988). 
A disadvantage of both simple random sampling and 
systematic sampling is that the sample sizes for rare 
classes will be small unless the overall sample size is 
very large. Fortunately, both simple random sampling 
and systematic sampling are equal probability sam- 
pling designs, so the proportion each class represents 
in the sample will be approximately equal to the pro- 
portion the class represents in the region mapped 
(Stehman and Foody, 2009; Jensen et al., 2012). 


In the hypothetical agriculture example, a systematic 
sample every 40 m in the x- and y-directions resulted in 
the allocation of only nine samples in the study area 
(Figure 13-4b). Eight of the nine samples were located 
on corn (89%). Soybeans received only one sample 
(11%). Decreasing the spacing of the systematic sample 
(e.g., to 20 m in x- and y-directions) would sample the 
soybean area more effectively. 


Stratified Random Sampling 


Analysts sometimes conduct stratified random sam- 
pling in which a minimum number of samples are allo- 
cated randomly to each stratum (i.e., land-cover 
category) after the thematic map has been prepared 
(Jensen and Shumway, 2010). For example, a stratified 
random sample based on 10 geographic regions was 
used to assess the accuracy of the U.S. National Land 
Cover Dataset (NLCD) and impervious surface (Wick- 
ham et al., 2010, 2013). 


Stratified random sampling involves two steps. First, 
the study area is subdivided into land-cover strata 
based on what is found in the remote sensing—derived 
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thematic map. A stratum is created by extracting only 
pixels (or polygons) associated with a specific class. 
Some analysts prefer to mask out all unwanted classes 
to create the stratum file. Sample locations are then 
randomly distributed throughout the geographic stra- 
ta. For example, all pixels (or polygons in a visual in- 
terpretation project) associated with forest were placed 
in a single stratum (4) shown in Figure 13-3c. A ran- 
dom number generator was then used to allocate a suf- 
ficient number of random samples to this forest 
stratum. Similarly, all pixels (or polygons) associated 
with corn cultivation were placed in stratum 2 in Fig- 
ure 13-3c. Random samples were then allocated 
throughout this stratum. Note that only strata 3 and 4 
were completely homogeneous in Figure 13-3c. Strata 
1 and 2 had stratum 4 (forest) areas within them. This 
is quite common. Note that the stratification process 
resulted in random samples being allocated to the for- 
est islands even though they existed within other strata. 


The advantage of stratified random sampling is that all 
strata (e.g., land-cover classes), no matter how small 
their proportion of the entire study area, will have sam- 
ples allocated to them for error evaluation purposes. 
Without stratification, it is often difficult to find suffi- 
cient samples for classes that occupy a very small pro- 
portion of the study area. The drawback with stratified 
random sampling is that one has to wait until the the- 
matic map has been completed in order to allocate 
samples to the various land-cover strata. The ground 
reference test information can rarely be obtained on 
the same day as the remote sensing data acquisition. 


Stratified random sampling can also be applied to 
products derived from remote sensing area-frame sam- 
pling as long as the various land-cover field polygon 
boundaries associated with the individual frames are 
kept. This allows ground reference test samples to be 
allocated within each stratum, as discussed (Wickham 
et al., 2010, 2013; Stehman et al., 2012). 


In the hypothetical agriculture example, stratified ran- 
dom sampling allocated 75% of the samples (9) within 
the corn region and 25% of them (3) within the soy- 
bean region as expected (Figure 13-4b). Both were al- 
located the requisite number of samples relative to the 
proportion of area they occupy in the study area. 


Stratified Systematic Unaligned 
Sampling 


Stratified systematic unaligned sampling combines ran- 
domness and stratification with a systematic interval. 
It introduces more randomness than just beginning 
with a random x, y coordinate for the first sample in 
each stratum. Figure 13-3d demonstrates how a strati- 
fied systematic unaligned sample is created. First, 


point A is selected at random. The x-coordinate of A is 
then used with a new random y-coordinate to locate B, 
a second random y-coordinate to locate E, and so on 
across the top row of the stratum under investigation. 
By a similar process the y-coordinate of A is used in 
combination with random x-coordinates to locate 
point C and all successive points in the first column of 
the stratum under investigation. Then, the random x- 
coordinate of C and y-coordinate of B are used to lo- 
cate D, of E and F to locate G, and so on until all strata 
have sample elements. 


Cluster Sampling 


Sometimes it is difficult to go into the field to random- 
ly selected locations and obtain the required informa- 
tion. Therefore, some have suggested that several 
samples can be collected at a single random location. 
Unfortunately, each pixel in the cluster is not indepen- 
dent of the others. Therefore, Congalton (1988) warns 
that clusters no larger than 10 pixels and certainly not 
larger than 25 pixels should be used because of the lack 
of information added by each pixel beyond these clus- 
ter sizes. 


Some combination of random and stratified sampling 
provides the best balance of statistical validity and 
practical application. Such a system may employ ran- 
dom sampling to collect some assessment data early in 
the project, and random sampling within strata should 
be used after the classification has been completed to 
ensure that enough samples were collected for each cat- 
egory and to minimize periodicity (spatial autocorrela- 
tion) in the data (Congalton, 1988). Ideally, the x, y 
locations of the reference test sites are determined us- 
ing global positioning system (GPS) instruments. 


Charleston, SC, Case Study: To demonstrate several of 
the concepts, consider the 407 reference pixels used to 
assess the accuracy of the land-cover map of Charles- 
ton, SC, derived from the Landsat Thematic Mapper 
imagery shown in Figure 9-18a. The accuracy was as- 
sessed based on stratified random sampling after clas- 
sification. The methods involved making five files, each 
containing only the pixels of a specific land-cover class 
(i.e., the classified land-cover pixels in each file were re- 
coded to a value of | and the unclassified background 
to a value of 0). A random number generator was then 
used to identify random x, y coordinates within each 
stratified file until a sufficient number of points was 
collected (1.e., at least 50 for each class). The result was 
a Stratified random sample of the five classes. All loca- 
tions were then visited in the field or evaluated using 
large-scale orthophotography acquired during the 
same month as the Landsat TM data collection. Cross- 
tabulating the information resulted in the creation of 
the error matrix shown in Table 13-2. 
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TABLE 13-2 Stratified random sample-based error matrix associated with the land cover classification map derived from 
Landsat Thematic Mapper data of Charleston, SC, shown in Figure 9-18a. The map has five classes and 407 ground reference 


samples were collected. 


Row total 
Xi+ 


Column total 73 60 


Xy j 


103 50 121 407 


Overall Accuracy = 382/407 = 93.86% 


Producer's Accuracy (omission error) 
Residential = 70/73 = 96% 4% omission error 
Commercial = 55/60 = 92% 8% omission error 


Wetland = 99/103 = 96% 4% omission error 
Forest = 37/50 = 74% 26% omission error 
Water = 20/22 = 100% 0% omission error 


User's Accuracy (commission error) 


Residential = 70/88 = 80% 20% commission error 
Commercial = 55/58 = 95% 5% commission error 
Wetland = 99/99 = 100% 0% commission error 
Forest = 37/41 = 90% 10% commission error 
Water = 121/121 = 100% 0% commission error 


where N = 407 


k 
by X jp = (70 + 55 + 99 + 37 + 121) = 382 


i=1 


i=1 


therefore K = 407 = — 36792 _ 155474 — 36792 
407° — 36792 


Computation of Kj; Coefficient of Agreement: K= 


k 
DY) Orjg XX pj) = BB x73) + (58 x 60)+ (99 x 103)+ (41 x 50}+ (121 x 121) = 36,792 


k k 
vy Xye— DY (js  ¥4,) 
i=1 i=1 
5 k 
N- DY (Kis X ¥4,) 


i=1 


Obtaining Ground Reference 
Information at Locations 


Using a Response Design 


The actual collection of ground reference samples for 
use in the accuracy assessment is technically referred to 


as the response design (Stehman and Czaplewski, 1998; 
Wickham et al., 2010). In effect, it is the data collection 
conducted in response to the sampling design. The re- 
sponse design can be subdivided into an evaluation 
protocol and a labeling protocol. The evaluation proto- 
col refers to selecting the support region on the ground 
(typically associated with a pixel or polygon) where the 
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ground information will be collected. Atkinson and 
Curran (1995) define this as the spatial support region 
consisting of “the size, geometry and orientation of the 
space on which an observation is defined.” Once the 
location and dimensions of the sampling unit are de- 
fined, the /abeling protocol is initiated and the sampling 
unit is assigned a hard or fuzzy ground reference label. 
This ground reference label (e.g., forest) is paired with 
the remote sensing—derived label (e.g., forest) for as- 
signment in the error matrix. 


> oa of Error Matrices 


After the ground reference test information has been 
collected from the randomly located samples, the test 
information is compared pixel by pixel (or polygon by 
polygon) with the information in the remote sensing— 
derived classification map. Agreement and disagree- 
ment are summarized in the cells of the error matrix as 
demonstrated in Table 13-2. Information in the error 
matrix may be evaluated using simple descriptive sta- 
tistics or multivariate statistical techniques. 


Descriptive Evaluation of Error 
Matrices 


There is no agreement on a standard approach to the- 
matic map accuracy assessment based on the analysis 
of error matrices. Each accuracy measure may be more 
relevant than others for a particular objective because 
different accuracy measures incorporate different in- 
formation about the error matrix. The use of different 
accuracy measures may result in different, possibly 
even conflicting, interpretations and _ conclusions 
(Foody, 2002; Liu et al., 2007). 


Liu et al. (2007) summarized 34 measures that were ap- 
plied to 595 error matrices found in the literature to as- 
sess classification accuracy. Their multidimensional 
scaling analysis provided detailed information about 
the suitability of the various measures. Only three 
heavily used measures are discussed here (e.g., Wick- 
ham et al., 2010, 2013). 


Nomenclature associated with a sample-based error 
matrix was presented in Table 13-1. The overall accura- 
cy of the classification map is determined by dividing 
the total number of correctly classified pixels (1.e., the 
sum of the diagonal in the error matrix) by the total 
number of pixels in the error matrix (V) (Story and 
Congalton, 1986; Congalton and Green, 2009): 


k 
YD ii 


Overall accuracy = = - , (13.4) 


Liu et al. (2007) found that this should be the primary 
measure for assessing the accuracy of an error matrix. 


Computing the accuracy of individual classes, howev- 
er, is more complex because the analyst has the choice 
of dividing the number of correct pixels in the class by 
the total number of pixels in the corresponding row or 
column. The total number of correct pixels in a class 
divided by the total number of pixels of that class as 
determined from the ground reference data (i.e., the 
column total) is called the producer’s accuracy (Story 
and Congalton, 1986; Congalton and Green, 2009): 


be 
Producer’s accuracy for class j = ao ; (13.5) 


an 


This statistic indicates the probability of a reference 
pixel being correctly classified and is a measure of 
omission error. 


If the total number of correct pixels in a class is divid- 
ed by the total number of pixels that were actually 
classified in that category, the result is a measure of 
commission error. This measure, called the user’s accu- 
racy or reliability (Story and Congalton, 1986; Con- 
galton and Green, 2009): 


Kees 

. IL 

User’s accuracy for class i = — 
Xe 

it 


(13.6) 


is the probability that a pixel classified on the map ac- 
tually represents that category on the ground. 


Sometimes we are producers of classification maps and 
sometimes we are users. Therefore, it is good practice 
to report all three accuracy measures: overall accuracy, 
producer’s accuracy, and user’s accuracy, because we 
never know how the classification will be used (Lu and 
Weng, 2007; Wickham et al., 2013; Pouliot et al., 
2014). 


For example, the remote sensing—derived error matrix 
in Table 13-2 has an overall classification accuracy of 
93.86%. However, what if we were primarily interested 
in the ability to classify just residential land use using 
Landsat TM data of Charleston, SC? The producer’s 
accuracy for this category was calculated by dividing 
the total number of correct pixels in the category (70) 
by the total number of residential pixels as indicated by 
the reference data (73), yielding 96%, which is quite 
good. We might conclude that because the overall ac- 
curacy of the entire classification was 93.86% and the 
producer’s accuracy of the residential land-use class 
was 96%, that the procedures and Landsat TM data 
used are adequate for identifying residential land use in 
this area. Such a conclusion could be a mistake. We 
should not forget the user’s accuracy, which is comput- 
ed by dividing the total number of correct pixels in the 
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residential category (70) by the total number of pixels 
classified as residential (88), yielding 80%. In other 
words, although 96% of the residential pixels were cor- 
rectly identified as residential, only 80% of the areas 
called residential are actually residential. 


A careful evaluation of the error matrix also reveals 
that there was confusion when discriminating residen- 
tial land use from commercial and forest land cover. 
Therefore, although the producer of this map can 
claim that 96% of the time a geographic area that was 
residential was identified as such, a user of this map 
will find that only 80% of the time will an area she or 
he visits in the field using the map will actually be resi- 
dential. The user may feel that an 80% user’s accuracy 
is unacceptable. 


Discrete Multivariate Techniques 
Applied to the Error Matrix 


Discrete multivariate techniques have been used to 
evaluate the accuracy of remote sensing—derived classi- 
fication maps and error matrices since 1983 (Congal- 
ton and Mead, 1983). The techniques are appropriate 
because remotely sensed data are discrete rather than 
continuous and are also binomially or multinomially 
distributed rather than normally distributed. Statistical 
techniques based on normal distributions simply do 
not apply. It is important to remember, however, that 
there is not a single universally accepted measure of ac- 
curacy, but instead there are a variety of indices, each 
sensitive to different features (Foody, 2002; Lu and 
Weng, 2007). 


It is useful to review several multivariate error evalua- 
tion techniques using the error matrix in Table 13-2. 
First, the raw error matrix can be normalized (stan- 
dardized) by applying an iterative proportional fitting 
procedure that forces each row and column in the ma- 
trix to sum to | (not shown). In this way, differences in 
sample sizes used to generate the matrices are eliminat- 
ed and individual cell values within the matrix are di- 
rectly comparable. In addition, because the rows and 
columns are totaled (i.e., the marginals) as part of the 
iterative process, the resulting normalized matrix is 
more indicative of the off-diagonal cell values (1.e., the 
errors of omission and commission). In other words, 
all the values in the matrix are iteratively balanced by 
row and column, thereby incorporating information 
from that row and column into each cell value. This 
process then changes the cell values along the major di- 
agonal of the matrix (correct classification), and there- 
fore a normalized overall accuracy can be computed 
for each matrix by summing the major diagonal and 
dividing by the total of the entire matrix. Therefore, it 
may be argued that the normalized overall accuracy is 
a better representation of accuracy than is the overall 


accuracy computed from the original matrix because it 
contains information about the off-diagonal cell values 
(e.g., Congalton, 1991; Stehman and Foody, 2009). 


Standardized error matrices are valuable for another 
reason. Consider a situation where analyst | uses clas- 
sification algorithm A and analyst 2 uses classification 
algorithm B on the same study area to extract the same 
four classes of information. Analyst A evaluates 250 
random locations to derive error matrix A and analyst 
B evaluates 300 random locations to derive error ma- 
trix B. After the two error matrices are standardized, it 
is possible to directly compare cell values of the two 
matrices to see which of the two algorithms was better. 
Therefore, the normalization process provides a conve- 
nient way to compare individual cell values between er- 
ror matrices regardless of the number of samples used 
to derive the matrix. 


Kappa Analysis 

Kappa analysis is a discrete multivariate technique of 
use in accuracy assessment. The method was intro- 
duced to the remote sensing community in 1981 and 
was first published in a remote sensing journal in 1983 
(Congalton, 1981; Congalton et al., 1983). 


Kyat Coefficient of Agreement Kappa computes a 
statistic, K, which is an estimate of Kappa. It is a mea- 
sure of agreement or accuracy between the remote 
sensing—derived classification map and the reference 
data as indicated by a) the major diagonal, and b) the 
chance agreement, which is indicated by the row and 
column totals (often referred to as marginals) (Paine 
and Kiser, 2003; Congalton and Green, 2009). K is 
computed: 


(13.7) 


where k is the number of rows (e.g., land-cover classes) 
in the error matrix, x;; is the number of observations in 
row i and column i, and x;, and x,; are the marginal 
totals for row i and column _/, respectively, and N is the 
total number of samples. K values >0.80 (1.¢e., >80%) 
represent strong agreement or accuracy between the 
classification map and the ground reference informa- 
tion. K values between 0.40 and 0,80 (1.e., 40% to 80%) 
represent moderate agreement. K values <0.40 (.e., 
<40%) represent poor agreement. 


Congalton and Green (2009) caution that if a stan- 
dardized error matrix is used that has a great many off- 
diagonal cells with zeros (i.e., the classification is very 
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good), then the normalized results may disagree with 
the overall classification accuracy and standard Kappa 
results. 


Charleston, SC, Case Study: The computation of the K 
statistic for the Charleston, SC, dataset is summarized 
in Table 13-2. The overall classification accuracy is 
93.86%, and the K statistic is 92.1%. The results are 
different because the two measures incorporated differ- 
ent information. The overall accuracy incorporated 
only the major diagonal and excluded the omission 
and commission errors. Conversely, K computation in- 
corporated the off-diagonal elements as a product of 
the row and column marginals. Therefore, depending 
on the amount of error included in the matrix, these 
two measures may not agree. Congalton (1991) sug- 
gests that overall accuracy, normalized accuracy, and 
K be computed for each matrix to “glean as much in- 
formation from the error matrix as possible.” Compu- 
tation of the K_ statistic may also be used 1) to 
determine whether the results presented in the error 
matrix are significantly better than a random result 
(1.e., a null hypothesis of K = 0), or 2) to compare two 
similar matrices (consisting of identical categories) to 
determine if they are significantly different. 


Savannah River Site, SC, Case Study: It is important to 
monitor the land cover of clay-capped hazardous 
waste sites on a repetitive basis to ensure the integrity 
of the clay caps (Jensen et al., 2003). This was accom- 
plished at the Westinghouse Savannah River Site near 
Aiken, SC, using DAIS 3715 hyperspectral imagery ob- 
tained on July 31, 2002, and a Spectral Angle Mapper 
classification algorithm. The accuracy of the classifica- 
tion was determined by first collecting detailed in situ 
land-cover reference data at the 98 locations shown in 
Figure 13-5a. All test sites were inventoried to within + 
30 cm using survey-grade GPS. The land cover at each 
of these locations was identified in the classification 
map shown in Figure 13-5b. The classification accura- 
cy results are summarized in Table 13-3. The overall 
classification accuracy was 89.79% with a K statistic of 
85.81%. 


Conditional K,,; Coefficient of Agreement The 
conditional coefficient of agreement (K,,) can be used 
to calculate agreement between the reference and re- 
mote sensing—derived data with chance agreement 
eliminated for an individual class for user accuracies 
using the equation (Paine and Kiser, 2003; Congalton 
and Green, 2009): 

k= N(x 5;) — 4 * x45) 


13.8 
*  N(xi4)— 4 * X4;) oe 


where x;; is the number of observations correctly classi- 
fied for a particular category (summarized in the diag- 
onal of the matrix), x;, and x,; are the marginal totals 
for row i and column i associated with the category, 
and JN is the total number of observations in the entire 
error matrix. For example, the conditional K, coeffi- 
cient of agreement for the residential land-use class of 
the Charleston, SC, dataset (Table 13-2) is: 


: _ 407(70)—(88 x 73) _ 28490-6424 _ 
K = = = 0.75 
Resid ~ 407(88)—(88x 73) 35816-6424 : 

This procedure can be applied to each land-cover class 

of interest. 


Kappa Debate Hundreds, perhaps even thousands of 
remote sensing investigations have used Kappa analy- 
sis to assess the accuracy of remote sensing—derived 
land-cover maps. It is important to note, however, that 
some scientists do not believe that Kappa analysis is 
the most appropriate measure for assessing the accura- 
cy of a remote sensing—derived thematic map. For ex- 
ample, Liu et al. (2007) evaluated 34 measures applied 
to error matrices and did not recommend the use of 
Kappa measures. Foody (2009) suggested that some of 
the arguments offered for the use of Kappa may be 
flawed and/or apply equally to other measures of accu- 
racy. Pontius and Millones (2011) evaluated five differ- 
ent Kappa indices including some put forth by Pontius. 
They suggested that the Kappa indices are misleading 
and/or flawed for practical applications. They then 
proposed the analysis of error matrices using two sum- 
mary parameters: quantity disagreement and alloca- 
tion. 


Conversely, Congalton (2005) and Congalton and 
Green (2009) believe that Kappa analysis has become a 
standard component of most accuracy assessments 
and is considered a required component of most digital 
image processing software programs. The remote sens- 
ing community does not always adopt every new accu- 
racy assessment approach that is put forth. It will be 
interesting to see what direction the remote sensing 
community takes. 


Fuzzification of the Error Matrix 


A hard classification system requires that all classes be 
mutually exclusive and exhaustive. Furthermore, the 
classes must usually be hierarchical, meaning that 
more detailed classes logically can be combined to pro- 
duce more general but useful classes. Some phenomena 
in the real world may in certain instances be considered 
pure or homogeneous (e.g., a monoculture corn field 
with complete canopy closure). Unfortunately, the 
world contains many phenomena that are not well 
mapped using hard classification logic. Sometimes a 
field or polygon contains multiple materials. These are 
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Accuracy Assessment In Situ Measurement Locations and Remote Sensing—Derived 
Classification Map of the Mixed Waste Management Facility on the Savannah River Site 


Location of in situ measurements 
acquired July 29-30, 2002, on 
A the Mixed Waste Management Facility } 


4 


_\ Vegetation classification based on analysis 
‘ie of DAIS 3715 hyperspectral data 
obtained on July 31, 2002 


y 


@ Bahia grass 


'@ Centipede 


@ Bare soil 
@ Concrete 


b. Classification map derived from Spectral Angle Mapper analysis of hyperspectral data. 


FIGURE 13-5 Classification accuracy assessment of a land-cover map of the Mixed Waste Management Facility on the Sa- 
vannah River Site in 2002 based on the analysis of DAIS 3715 hyperspectral data and in situ ground reference data. a) The lo- 
cation of Bahia grass and Centipede grass in situ measurement locations. b) Remote sensing—derived classification map. 


often referred to as endmembers (e.g., corn inter- 
cropped with hay, bare soil between rows of corn, 
shadows, etc.). In this case, per-pixel or polygon class 
membership may be a matter of degree or percentage 
rather than assignment to a hard, mutually exclusive 
class (Foody, 2002). For example, if a single field or 


polygon contains 49% corn and 51% hay, should it re- 
ally be labeled a hay field or should it be called some- 
thing else? In fact, many natural landscapes often 
grade gradually into one another at the interface (e.g., 
it is common to find grassland grading into scrub/ 
shrub and then scrub/shrub transitioning into forest). 
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TABLE 13-3 Accuracy assessment of the classification map derived from hyperspectral data of the Mixed Waste Manage- 


ment Facility on the Savannah River Site (Jensen et al., 2003). 


Row total 
Xi+ 


Column total 
Xj 


Overall Accuracy = 88/98 = 89.79% 


Producer's Accuracy (omission error) 

Bahia grass = 31/37 = 84% 16% omission error 
Centipede = 7/9 = 78% 22% omission error 
Bare soil = 30/30= 100% 0% omission error 
Concrete = 20/22= 91% 9% omission error 


User's Accuracy (commission error) 

Bahia grass = 31/33 = 94% 6% commission error 
Centipede = 7/15 = 47% 53% commission error 
Bare soil= 30/30 = 100% 0% commission error 
Concrete = 20/20= 100% 0% commission error 


where N = 98 


k 
> X;,=(81+7 + 30 + 20) = 88 


i=1 


i=1 


therefore K = 5 
98° — 2696 


Computation of Kj; Coefficient of Agreement 


k 
> (Xj4 X X4;) = 3 x 37) + (15 x 9)+ (30 x 30)+ (20 x 22) = 2696 


5928 pe gig 
Fagg ee ehM 


~ _ 98(88)- 2696 _ 8624-2696 _ 
9604 — 2696 


Gopal and Woodcock (1994) were among the first to 
suggest that fuzzy logic be used to introduce real-world 
fuzziness into the classification map accuracy assess- 
ment process. Instead of a right or wrong (binary) 
analysis, map labels were considered partially right (or 
wrong), generally on a five-category scale. Various sta- 
tistics were then derived from the fuzzy information, 
including Max (M), the number of sites with an abso- 
lutely right answer (rank of 5); Right (R), the number 


of sites with a reasonable, good, or absolutely right an- 
swer (ranks 3, 4, and 5); and Increase (R—M), which is 
the difference between the Right and Max functions 
(Jacobs and Thomas, 2003). These were used to pro- 
duce more sophisticated fuzzy logic-based accuracy 
assessment measures. 


NIMA GeoCover Land-cover Case Study: Green and 
Congalton (2003) used fuzzy logic during the phase 
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FIGURE 13-6 A template used for the introduction of fuzzy reference test information in an accuracy assessment of NIMA- 
sponsored Landsat GeoCover land-cover maps. The ground reference test polygon under consideration (21) has been as- 
signed the deterministic class of deciduous forest. However, the analyst believes it could also be evergreen forest. The analyst 
is sure it is not one of the other categories. These data can be used to create an error matrix that contains deterministic as 
well as fuzzy accuracy assessment information (based on Green and Congalton, 2003). 


when ground reference information were collected and 
compared it with the remote sensing-derived classified 
map results. Their goal was to assess the accuracy of a 
NIMA-sponsored global Landsat (30 x 30 m) Geo- 
Cover land-cover map (Green and Congalton, 2003). It 
was not practical to visit on the ground all the sites se- 
lected in the worldwide stratified random sample. 
Therefore, NIMA provided high spatial resolution na- 
tional technical means (NTM) imagery of the selected 
sample sites. These data were interpreted and repre- 
sented the ground reference test information. 


Experienced interpreters used a version of the fuzzy ac- 
curacy assessment form shown in Figure 13-6 to enter 
their ground reference test information. Each ground 
reference test sample location in the NTM imagery was 
evaluated for the likelihood of being identified as each 
of the possible cover types. The analyst first deter- 
mined the most appropriate label for the sample site, 
e.g. deciduous forest, which was then entered in the 
“classification” column in the form (Figure 13-6). Af- 
ter assigning the deterministic label for the sample, the 
remaining possible map labels were evaluated as 
“good,” “acceptable,” or “poor” candidates for the 
site’s label. For example, a sample site might fall near 
the classification scheme margin between deciduous 
forest and evergreen forest. In this instance, the analyst 
might rate evergreen forest as acceptable, but scrub/ 
shrub and the other categories as “poor.” As each site 


was interpreted, the deterministic and fuzzy assess- 
ment reference labels were entered into the error matrix 
(Table 13-4) (Green and Congalton, 2003). 


Nondiagonal cells in the error matrix (Table 13-4) con- 
tain two tallies that can be used to distinguish class la- 
bels that are uncertain or that fall on class margins 
from class labels that are probably in error (Green and 
Congalton, 2003; 2005). The first number represents 
sites where the map label matched a “good” or “ac- 
ceptable” reference label in the fuzzy assessment. 
Therefore, even though the class label was not consid- 
ered the most appropriate, it was considered acceptable 
given the fuzziness of the classification system and the 
minimal quality of some of the ground reference test 
data. These sites are considered a “match” for estimat- 
ing fuzzy assessment accuracy. The second number in 
the cell represents those sites whose map class label was 
considered poor (i.e., an error). 


The fuzzy assessment overall accuracy was estimated 
as the percentage of sites where the “good” or “accept- 
able” reference label(s) matched the map label. Individ- 
ual class accuracy was estimated by summing the 
number of matches for that class’s row or column di- 
vided by the row or column total. Class accuracy by 
column represented producer’s class accuracy. Class 
accuracy by row represented user’s accuracy (Green 
and Congalton, 2003; Congalton and Green, 2009). 
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TABLE 13-4 An error matrix that contains deterministic and fuzzy information used to assess the classification accuracy of 
NIMA-sponsored Landsat GeoCover thematic map products derived from Landsat imagery (based on Green and Congalton, 
2003). 


User's User's 
Row Total Row Total 
Deterministic Fuzzy 
48/113 72/113 
(42.5%) (63.7%) 
0,0 (0),1) 17/26 21/26 
(65.4%) (80.8%) 
2,0 0,1 N55 8,1 0,0 0,0 2,2 0,0 15/31 27/31 
(48.4% (87.1%) 
0,1 0,0 Syl 14 0,0 0,0 3,0 0,0 14/24 22/24 
(58.3%) (91.7%) 
0,0 0,0 0,2 0,0 0 0,0 0,1 0,0 0/3 0/3 
(0%) (0%) 
0,0 0,0 0,0 0,0 0,0 20 2,0 0,0 20/22 22/22 
(90.9%) (100%) 
0,1 0,1 TA) 18,6 0,0 2,0 29 1,2 29/82 57/82 
(35.4%) (69.5%) 
0,0 0,0 0,0 0,0 0,0 0,0 0,0 8 8/8 8/8 
(100%) (100%) 
Producers 48/56 17/50 15/47 14/50 na 20/24 29/51 8/33 
Column Total (85.7%) (34%) (31.9%) | (28%) na (83.3%) | (56.9%) | (24.2%) 
Deterministic 
Producers 54/56 41/50 27/47 40/50 na 22/24 36/51 10/33 
Column Total (96.4%) | (82%) | (57.4%) | (80%) na (91.7%) | (70.6%) | (33.3%) 
Fuzzy 
Overall deterministic accuracy = 151/311 = 48.6% 
Overall fuzzy accuracy = 230/311 = 74% 


Overall accuracy for the deterministic components of 
the matrix was 48.6% (151/311). The deterministic sta- 
tistic ignored any variation in the interpretation of ref- 
erence data and the inherent fuzziness at class 
boundaries. Including the “good” and “acceptable” 
ratings, overall fuzzy accuracy was 74% (230/311). 


Gopal and Woodcock (1994) introduced ground- 
breaking methods for quantifying accuracy when the 
reference classification is fuzzy and the map classifica- 
tion is hard (crisp) (Stehman, 2009). Pontius and 
Cheuk (2006) extended the error matrix concept to 
provide descriptive accuracy measures when both the 
map and the reference classifications are fuzzy. Gomez 
et al. (2008) developed new accuracy measures that 
provide overall, producer’s, user’s or Kappa statistics 
when working with fuzzy or hard (crisp) error matrices. 


An error matrix and the estimates of classification ac- 
curacy derived from it provide no information on the 
spatial distribution of the error (Foody, 2005). Comber 
et al. (2012) used geographically weighted regression 
approaches to model the spatial variations in the accu- 
racy of both hard (crisp) Boolean and soft (fuzzy) 
land-cover classes. 


Change Detection Map 
Accuracy Assessment 


van Oort (2007) and Warner et al. (2009) suggest that 
there are three basic types of error matrices reported 
for change detection products: 


1. Accuracy assessment of the classification of each 
independent date. 
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TABLE 13-5 Hypothetical accuracy assessment of a remote sensing—derived thematic map with just three classes: Buildings, 


Roads, and Other (based on Jensen et al., 2012). 


Ground Reference Information 
Class 1 to k (j columns) 
Building Road Other Row total 
Building 49 4 5 58 
Ma 
ide Road 5 54 4 63 
1tok 
(i rows) Other 6 6 59 71 
Column total 60 64 68 192 


Overall accuracy = 162/192 = 84.4% 


Producer's accuracy Buildings = 49/60 = 82% 


Roads = 54/64 = 84% Other = 59/68 = 87% 


User's accuracy Buildings = 49/58 = 82% 


Roads = 54/63 = 84% Other = 59/71 = 87% 


Kappa Coefficient of Agreement (I< ) = [192(162) — 12,340] / (36864 — 12,340) = 76.5% 


2.A complete change transition error matrix 
consisting of all possible change and no-change 
classes and their potential confusion with each 
other. 


3. Accuracy assessment of a simple binary change/no- 
change classification. 


Assessing the Accuracy of the 
Individual Thematic Maps used in a 
Change Detection Study 


This type of accuracy assessment is performed for each 
of the classification maps used in the change detection 
study. Table 13-5 demonstrates how a simple 3 x 3 er- 
ror matrix is sufficient for assessing the accuracy of a 
single-date three-class map consisting of Building, 
Road, and Other land cover. Unfortunately, analysis of 
the accuracy of the individual date thematic maps does 
not provide any accuracy information about the 
change map produced from the intersection of the two 
individual date thematic maps. 


Assessing the Accuracy of a 
“From-—To” Change Detection Map 


Assessing the accuracy of a change detection map con- 
sisting of “from—to” classes is no trivial matter (Khor- 
ram et al., 1999; Foody, 2002; Jensen and Im, 2007). 
For example, when evaluating the thematic accuracy of 
a change detection map with the same three classes 
(Building, Road, Other) derived from two dates of 1m- 
agery, there is now the possibility of 81 change classes 
in the change map as shown in Table 13-6. Thus, if 


there are k classes in each of the maps used to detect 
change, then the total number of change classes equals 
k x k? (Warner et al., 2009). This significantly compli- 
cates the process of collecting a sufficient number of 
ground reference samples to populate the error matrix 
(Congalton and Green, 2009; Warner et al., 2009). 


Response Design 

The response design considerations (e.g., per pixel, 
polygon) used to assess the accuracy of a single-date 
thematic map hold true when analyzing the informa- 
tion in a change map (Jensen et al., 2012). However, 
the user must be particularly careful about specifying 
the desired acceptable level of accuracy for change de- 
tection map products. They may not be the same as 
those used when assessing the accuracy of a single date 
thematic map. 


Sampling Design 

The same general rule-of-thumb about the required 
number of samples unfortunately holds true when as- 
sessing the accuracy of a change map, 1.¢., approxi- 
mately 50 samples per class. However, when assessing 
the accuracy of a simple 3-class change map, the ana- 
lyst must now somehow determine the land cover on 
450 random locations (9 change classes x 50 samples 
per class) (Congalton and Green, 2009). This is a 
daunting task. Also, some of the possible “from-to” 
change classes have a very low probability of occur- 
ring, such as a change from “Building” to “Other” un- 
less the building was destroyed. Nevertheless, the 
methods for obtaining the requisite reference samples 
to document the accuracy of the change detection map 
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TABLE 13-6 A change detection error matrix for the same three map categories [Building (B), Road (R), and Other (O)] that 
were used in the single-date accuracy assessment (based on Macleod and Congalton, 1998; Jensen et al., 2012). 


Ground Reference Information 
Class 1 to k (j columns) 


BB RR (ete) BR BO RB RO OB OR Row 
total 
BB 
RR 
fete) 
Change BR [ 
Detection 
Map Class BO |] 
1tok 
r pal 
08 ina 
Column 
total 


are the same as those used when assessing the accuracy 
of an individual thematic map, including: random sam- 
pling, stratified sampling, systematic sampling, and 
cluster sampling (Jensen et al., 2012). 


Analysis 

Given careful planning and sufficient resources to ob- 
tain sufficient ground reference points at two dates in 
time, it is possible to populate the change detection er- 
ror matrix with measurements. The major diagonal of 
the change matrix is highlighted in black in Table 13-6 
and represents when the Date | classification of a pixel 
or polygon agrees with the Date 2 classification of the 
same pixel or polygon (e.g., it was a “Building” on 
Date 1 and a “Building” on Date 2 [i.e., BB]). Similar- 
ly, if the pixel or polygon on Date 1 was “Other” and 
“Other” on Date 2, then a hash mark would be placed 
in the OO cell as shown. 


The matrix also accounts for when the land cover 
changes. For example, if “Other” (e.g., vegetation) was 
truly present on Date | and was transformed into a 
“Building” on Date 2, then a hash mark would be 
placed in the OB diagonal cell as shown in Table 13-6. 
Conversely, if the “Other” pixel or polygon present on 
Date | was erroneously classified as “Road” instead of 
“Building” on the Date 2 map, then a hash mark 
would be placed in the off-diagonal OR cell of the ma- 
trix as shown. All hash marks placed in off-diagonal 
cells are change detection map classification errors. Af- 
ter the change detection error matrix is sufficiently 
populated with unbiased information obtained from 


the two classification maps, then it is possible to com- 
pute the overall accuracy, producer’s accuracy, and us- 
er’s accuracy as previously discussed. 


The complete change transition error matrix (e.g., Ta- 
ble 13-6) provides the most useful information. Howev- 
er, it is only occasionally included in change studies 
because of the challenge of collecting sufficient data 
for so many categories especially when many of the cat- 
egories are very rare occurrences. This makes simple 
random sampling inefficient. The problem can be im- 
proved somewhat using stratified sampling. 


Assessing the Accuracy of a Binary 
Change Detection Map 


Many change detection studies only attempt to deter- 
mine if one class, e.g., forest, has changed from Date 1 
to Date 2 (Im et al., 2008). This is called binary change 
detection (Im et al., 2009, 2011). Both the Date 1 and 
Date 2 remote sensing—derived thematic maps actually 
have two classes: forest and non-forest, i.e., a binary 
classification. When the two binary thematic maps are 
compared using post-classification comparison change 
detection (refer to Chapter 12) the result is a change 
detection map that can be analyzed using multiple-date 
ground reference information as shown in Table 13-7 
(Khorram et al., 1999; Foody, 2002). 


Foody (2010) suggests that this binary error matrix 
may be used to describe the sensitivity, specificity, and 
prevalence of the remote sensing—derived change map. 
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TABLE 13-7 A binary error matrix that can be used to com- 
pute the characteristics of a remote sensing—derived change 
detection map (based on Foody, G. M., 2010, “Assessing the 
Accuracy of Land Cover Change with Imperfect Ground Refer- 
ence Data,” Remote Sensing of Environment, 114:2271-2285). 


Multiple-date Binary 
Ground Reference 
Information 


Sensitivity is the proportion of cases correctly classified 
as having changed (Foody, 2010): 


Sensitivity = ae = (13.9) 


Specificity is the proportion of cases correctly predict- 
ed to have not changed: 


a = 


Specificity = hed : 


(13.10) 
Sensitivity and specificity parameters represent the 
producer’s accuracy for the change and no-change 
classes, respectively. 


Evaluating the binary error matrix horizontally allows 
the predicted positive value (Foody, 2010): 


: __a@ _a 
Predicted, ra (13.11) 
and the predicted negative value: 
: . 8:8 
Predicted, = a (13.12) 


to be determined. The positive and negative predicted 
values for the change detection map represent the us- 
er’s accuracy (Lu and Weng, 2007; Liu et al., 2007) for 
the change and no-change classes, respectively. 


Prevalence (8) is computed using (Foody, 2010): 


+ 
Prevalence (0) = —2=c =f 


atb+ct+d N Oe?) 


Foody (2010) used these measures to compute the im- 
pact that error in ground reference data has on the ac- 
curacy of remote sensing—derived land-cover change 
detection maps. 


Olofsson et al. (2014) identified additional good prac- 
tices for assessing the accuracy and estimating the area 
of change in a land-cover change map. They provided 
recommendations for: 1) the sampling design, 2) the re- 
sponse design, and 3) the analysis that should take 
place. 


Assessing the Accuracy of an 
Object-Based Image Analysis (OBIA) 
Classification Map 


Lizarazo (2014) introduced a method to assess the ac- 
curacy of land-cover maps produced using object- 
based image analysis (OBIA). It is based on the use of 
classified “objects” in the map compared with refer- 
ence “objects.” Four STEP metrics are computed, in- 
cluding: 1) shape similarity (S); 2) theme similarity (T); 
3) edge similarity (E); and 4) position similarity (P). In- 
dividual objects’ similarity metrics are grouped by the- 
matic class and expressed in the integrated STEP 
similarity matrix. The STEP indices and matrices can 
provide useful information about thematic and geo- 
metric accuracies of object-based image classifications. 
Moller et al. (2014) provide a framework for assessing 
the geometric accuracy of objects classified in an 
OBIA-derived classification map. 


Geostatistical Analysis in 
Support of Accuracy 
Assessment 


The emphasis in the previous sections on error evalua- 
tion is primarily concerned with the accuracy of the- 
matic (categorical) information. However, the Earth’s 
surface and remotely sensed images of that surface also 
have distinct spatial properties. Once quantified, these 
properties can be used for many tasks in remote sens- 
ing, including image classification and the sampling of 
both the image and the ground reference test data 
(Curran, 1988). 


Where there is spatial dependence (i.e., autocorrela- 
tion) in a remote sensing-derived land-cover map, the 
number of sample points required to achieve a given 
level of confidence might be much fewer if the study is 
based on a systematic sample survey instead of a com- 
pletely random survey (Curran, 1988). Constructing 
the empirical semivariogram model and evaluating its 
range and sill may make it possible to identify the criti- 
cal autocorrelation characteristics of the dataset and 
design a sampling scheme that ensures that neighbor- 
ing sample points are as far from one another as is 
practical for a fixed sample size and area (Van der 
Meer, 2012). This type of sampling scheme would min- 
imize the duplication of information that often occurs 
in random sampling where some sample points are in- 
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FIGURE 13-7 a) A hypothetical map containing crop polygons. b) Semivariogram of the distribution of polygons. c) Predict- 
ed crop polygons. The semivariogram information can be used to improve accuracy assessment sampling design. 


in random sampling where some sample points are in- 
evitably close to one another. Curran (1988) describes 
how to test various grid-spacing designs that allow 
sampling error and sample size to be considered in a 
spatial context. He suggests that a well-designed sys- 
tematic survey based on geostatistical analysis can in- 
crease the precision over random sampling and 
decrease the sometimes impossibly large sample sizes 
associated with random sampling (Curran and Wil- 
liamson, 1986). 


For example, consider the hypothetical map of remote 
sensing—derived agricultural crop polygons shown in 
Figure 13-7. Its semivariogram (discussed in Chapter 
4) and predicted image are also provided. The geosta- 
tistical spherical kriging process did a good job of pre- 
dicting the original surface. Plus, we now have detailed 
information about the agricultural field autocorrela- 
tion present in the dataset. If this type of information 
were available over an entire region, it would be possi- 
ble to identify the optimum sampling frequency that 
should be used for a) identifying the optimum spatial 
resolution of the remote sensing system, b) the spacing 
of individual frames of imagery if the study involves a 
sampling scheme rather than wall-to-wall mapping 
and/or c) allocating ground reference test sites to per- 
form a classification accuracy assessment. 


Jacobs and Thomas (2003) used fuzzy set analysis and 
geospatial statistics (kriging) to create a spatial view of 
the accuracy of an Arizona GAP Analysis map. Kyria- 
kidis et al. (2005) described geostatistical approaches 
for mapping thematic classification accuracy. Zhu and 
Stein (2006) and Van der Meer (2012) review how geo- 
statistics can be used to design sampling schemes and, 
prior to actually sampling these in the field, determine 
the kriging estimation variance that will be obtained. 
Burnicki (2011) used geostatistical analysis to examine 
the spatial and temporal patterns of error in time-series 
individual maps and the change detection map. 


A Metadata and Lineage 
Information for Remote 
Sensing-Derived Products 


It is becoming increasingly important to document a) 
all of the information about the creation of an individ- 
ual remotely sensed image, and b) the various proce- 
dures and processes applied to the imagery to create an 
output product and assess its accuracy. 


Individual Image Metadata 


The U.S. government now requires that all imagery 
collected for government use contain detailed metada- 
ta (data about data) as to how the images were collect- 
ed. Detailed metadata standards have been developed 
for raster image datasets by the U.S. Federal Geo- 
graphic Data Committee (FGDC). This type of infor- 
mation allows the image analyst and decision-maker to 
document the source of the imagery and all of its in- 
herent characteristics. Metadata standards are an es- 
sential component of national and global spatial data 
infrastructures (SDI) (Jensen et al., 2002). 


Lineage of Remote Sensing—Derived 
Products 


It is not sufficient to collect only image metadata. It is 
usually necessary to carefully document every opera- 
tion that is performed on the remote sensing data that 
results in an output product such as a land-use or land- 
cover map. This is commonly referred to as lineage doc- 
umentation. Unfortunately, manual bookkeeping of 
the processes used to create a final product is cumber- 
some and rarely performed. Some digital image pro- 
cessing systems do provide history or audit files to 
keep track of the iterations and operations performed. 
However, none of these methods is capable of fulfilling 
the information requirements of a true lineage report 
that itemizes the characteristics of both the image and 
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cartographic sources, the topological relationships 
among sources, intermediate and final product layers, 
and a history of the transformations applied to the 
sources to derive the output products (Lanter, 1991). 


Lineage information should be included in quality as- 
surance reports associated with every remote sensing— 
derived thematic product, including: 


* asummary of all image and ancillary source materi- 
als (e.g., soil and geology maps, digital terrain mod- 
els) used to derive the thematic information; 

* geoid, datum, and map projection information used 
to create the thematic information; 

* geometric rectification and radiometric correction 
parameters; 

¢ the processing steps used to transform the remotely 
sensed data into information; 


¢ the methods and results of the accuracy assessment; 

¢ the location of the archived original, interim, and 
final datasets; 

* procedures for accessing the archived information; 
and 


* geometric and thematic attribute accuracy assess- 
ment, if available. 


Quality assurance is an important part of life today. 
Image analysts who extract thematic information from 
remotely sensed data add value and rigor to the prod- 
uct by documenting its lineage. 
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APPENDIX: SOURCES OF IMAGERY 
AND OTHER GEOSPATIAL 
INFORMATION 


tion is one of the most important and challeng- 

ing activities of remote sensing and GIS 
investigations. Chapter 2 reviewed numerous types of 
remote sensing data acquired using airborne and satel- 
lite remote sensing systems. This Appendix provides 
more detailed information about how to access select- 
ed types of remote sensing and other types of geospa- 
tial data using the Internet. The datasets listed are not 
exhaustive, but will hopefully provide sufficient infor- 
mation to evaluate and obtain some of the most com- 
monly used types of remote sensing and other types of 
geospatial data. 


O btaining timely, accurate, geospatial informa- 


This Appendix begins by introducing four federal geo- 
spatial data search engines and image repositories. 
Next, three representative commercial geospatial data 
search engines and/or repositories are reviewed. The 
remainder of the Appendix is devoted to a discussion 
of selected: 1) thematic datasets (e.g., elevation, hydrol- 
ogy, land use/land cover), 2) sources of public remote 
sensor data, and 3) sources of commercial and interna- 
tional remote sensor data. 


{4 Table of Contents 


Federal Image and Geospatial Data 
Search Engines and Repositories 


¢ EarthExplorer (USGS) 

¢ The National Map (USGS) 

¢ Global Visualization Viewer—Glovis (USGS) 
* Data.gov (USGS) 


Source: NOAA 


Commercial Image and Geospatial 
Data Search Engines and/or 
Repositories 


* Google earth 
* Microsoft bing 
¢ Esri ArcGIS Online Map and Geoservices 


Digital Elevation Data 

¢ DRG—Digital Raster Graphics (USGS) 

* GTOPO30—Digital elevation model of the world 
(USGS) 

« NED—National Elevation Dataset (USGS) 


¢ Topographic and Bathymetric Information (USGS, 
NOAA) 


¢ Topographic Change Information (USGS) 


¢ SRTM—Shuttle RADAR Topography Mission 
(NASA JPL) 


¢ ASTER Global Digital Elevation Map (METI and 
NASA) 


¢ NEXTMap World 30 Digital Surface Model (Inter- 
map, Inc.) 


Hydrography Data 
« NHD—National Hydrologic Database (USGS) 


« EDNA—Elevation Derivatives for National Appli- 
cations (USGS) 
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USGS EarthExplorer User Interface 


— 


= USGS Ss eas 


Geographic Search Area 


a. 


Results of a Search for Landsat 5 Imagery 


Browse image of Landsat 5 data collected June 1, 2008 


S 


Charleston, SC 


RGB = 
bands 7,4,3 


b. 


FIGURE A-1 a) The USGS EarthExplorer user interface (http://earthexplorer.usgs.gov/). The geographic search area is avail- 
able online. A search for Landsat 4 and 5 imagery of Charleston, SC, from 2005 to 2010 in June was selected. b) This June 1, 
2008, Landsat 5 image was one of several images that met the search criteria (imagery courtesy of NASA). 


Land Use/Land Cover and 
Biodiversity/Habitat Data 
« NLCD—National Land Cover Dataset (USGS) 


« C-CAP—Coastal Change Analysis Program 
(NOAA) 


¢ GAP Analysis Program (USGS) 


¢ NWI—National Wetlands Inventory (USFWS) 
« NPN—USS. National Phenology Network 


Population Demographic Data 


¢ MAF/TIGER—Master Address File/Topologically 
Integrated Geographic Encoding and Reference 
System (Bureau of the Census) 


« 2010 Census Demographic Data (Bureau of the 
Census) 


¢ LandScan (Oak Ridge National Laboratory) 


Remote Sensor Data—Public 


¢ Selected publicly available sources of analog and 
digital remote sensor data. 


Remote Sensor Data—Commercial 
and International 


¢ Selected commercial and international sources of 
analog and digital remote sensor data. 


Federal Geospatial Data 
Search Engines and 
Repositories 


A powerful characteristic of federal geospatial data re- 
positories is that the data are in the public domain and 
available to the general public at no expense. Only a se- 
lect few of the many federal image and geospatial re- 
positories are introduced here. 


USGS EarthExplorer 


One of the best places to look for remote sensing data 
and various remote sensing—derived products is at the 
U.S. Geological Survey’s EarthExplorer website (http:// 
earthexplorer.usgs.gov/). This site allows users to select 
a geographic place or region by typing in the name of a 
location, entering geographic coordinates, or drawing 
on a map (Figure A-la). Once the geographic area has 
been identified, the user can identify specific types of 
data from the menu. This includes access to digitized 
aerial photography (discussed below), AVHRR, Cali- 
bration/Validation Reference Sites, Commercial (IKO- 
NOS-2), Declassified data, Digital Elevation (ASTER 
Global DEM, GMTED2010, GTOPO30, GTOPO30 
HYDRO IK, SRTM, SRTM Void Filled), Digital Line 
Graphs, EO-1 (ALI, Hyperion), Forest Carbon Sites, 
Landsat (Global Land Survey, Archive, CDR, Legacy, 
MRLC), LiDAR, NASA SPDAAC Collections, Orb- 
view-3, RADAR (SIR-C), Vegetation monitoring, HC- 
MM, and Land Cover (NLCD 1992, 2001, 2006). For 
example, the user interface in Figure A-la was used to 
search for Landsat 5 Thematic Mapper imagery of 
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The National Map 


Charleston, SC 


FIGURE A-2 The National Map user interface is especially useful for locating information about geographic boundaries, ter- 
rain elevation, geographic names, hydrography, land cover, orthoimagery, structures, and transportation (http://nationalmap 
. gov/ vever .html) . This is a display of 1& 1 ft. orthoimagery of Charleston, SC (aerial phot ography courtesy of the U.S. Geo- 


logical Survey). 


Charleston, SC, collected in June from 2005 to 2010. 
One of the results of the search is shown in Figure A- 
1b. The user may discard the search or order a digital 
copy of the June 2, 2008, Landsat image. 


From 1972 to 2005, the USGS Earth Resources Obser- 
vation and Science (EROS) Center provided remote 
sensing film-based products to the public. EROS is 
home to an archive of 12 million frames of analog pho- 
tography ranging from 1937 to the present. The archive 
contains analog collections from both aerial and satel- 
lite platforms including programs such as the National 
High Altitude Program (NHAP), National Aerial Pho- 
tography Program (NAPP), U.S. Antarctic Resource 
Center (USARC), Declassified 1 (CARONA, AR- 
GON, and LANYARD), Declassified 2 (KH-7 and 
KH-9), and Landsat (1972 — 1992, Landsat 1-5). Since 
2004, EROS has been digitally scanning the analog col- 
lection at resolutions ranging from 64 Xm (400 dpi) to 
25 Km (1,000 dpi) depending upon the severity of the 
vinegar syndrome (a slow form of chemical deteriora- 
tion that causes the film to shrink, buckle, and become 
unusable). The digitized imagery is available at no cost 
through the EarthExplorer interface (Moe and Longh- 
enry, 2013). 


Users must register with EarthExplorer to save search- 
es and order data. Some digital datasets can be down- 
loaded immediately. Other datasets take a few hours or 
a couple of days for the data to be extracted and then 
made available to the user via FTP. Normally, users are 
not sent to additional sites to order data. This is an ad- 
vanced search engine that is very useful for those who 
know the type and characteristics of the imagery and/ 
or other geospatial data they are looking for. 


USGS The National Map 


Managed by the National Geospatial Program (NGP), 
The National Map viewer (Figure A-2) provides data 
visualization and download of all eight National Map 
datasets, including: geographic boundaries, elevation, 
geographic names, hydrography, land cover, orthoim- 
agery, structures, transportation, and US Topo prod- 
ucts (Carswell, 2011; USGS NGP, 2014). The majority 
of The National Map effort is devoted to acquiring and 
integrating medium-scale (nominally 1:24,000-scale) 
geospatial data for the eight base layers from a variety 
of sources and providing access to the resulting seam- 
less coverages of geospatial data. 


An orthoimage is remotely sensed image data in which 
displacement of features in the image caused by terrain 


586 INTRODUCTORY GEOGRAPHIC INFORMATION SYSTEMS 


Global Visualization Viewer (Glovis) Interface 
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FIGURE A-3 The Global Visualization Viewer (Glovis) user interface provides information about the availability of the follow- 
ing types of imagery: aerial (NHAP, NAPP), ASTER, EO-1 (Advanced Land Imager, Hyperion), Landsat archive, Landsat Global 
Land Survey, Landsat MRLC collections, Landsat Legacy Collections, MODIS (Aqua, Terra, combined), and Terra Outlook 

(http://glovis.usgs.gov/). This is a display of high spatial resolution NAPP aerial photography of Hilton Head, SC, collected on 
January 8, 2006. The image can be downloaded directly from the Glovis website (aerial phot ography courtesy of U.S. Geolog- 


ical Survey). 


relief and sensor orientation have been mathematically 
removed. Orthoimagery combines the image charac- 
teristics of a photograph with the geometric spatial ac- 
curacy and reliability of a planimetric map. USGS 
digital orthoimage resolution may vary from 6 in. to 1 
m. The National Map provides free downloads of pub- 
lic domain, 1 X 1 m orthoimagery for the contermi- 
nous U.S. with many urban areas and other locations 
at 2-ft. or finer resolution. For example, 1 X 1 ft. spatial 
resolution orthoimagery of Charleston, SC, in The Na- 
tional Map database is displayed in Figure A-2. 


The National Map viewer also allows visualization and 
identification queries (but not downloads) of other da- 
ta, including: Ecosystems, Protected Areas, Gap Anal- 
ysis Program Land Cover, Hazards, Weather, 
Wetlands, Public Land Survey System, and National 
Park Service Boundaries. To research and download 
historical USGS data, such as Digital Orthophoto 
Quadrangles and Digital Line Graphs, or to access 


specific LiDAR point cloud data, use the USGS Earth- 
Explorer website. 


USGS Global Visualization Viewer 


The USGS Global Visualization Viewer (Glovis) is a 
straightforward tool for searching and ordering specif- 
ic types of satellite and aerial imagery. The types of 
data that can be searched include aerial (NHAP, 
NAPP), ASTER, EO-1 (Advanced Land Imager, Hy- 
perion), Landsat archive, Landsat Global Land Sur- 
vey, Landsat MRLC collections, Landsat Legacy 
Collections, MODIS (Aqua, Terra, combined), and 
Terra Outlook. 


A search for National Aerial Photography Program 
(NAPP) imagery of Hilton Head, SC, is shown in Fig- 
ure A-3. The color-infrared aerial photograph was ob- 
tained on January 8, 2006. If desired, this image can be 
downloaded directly from the Glovis interface. 
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FIGURE A-4 The Data.gov user interface (http://catalog.data.gov/dataset) allows the user to filter by location and then 
search a list of datasets by type. In this example, the user specified the “geospatial dataset type” and requested information 
on all the “Orthoimagery” for Charleston, SC. The results of the search identified 43 orthoimagery datasets available for the 
study area. The user can evaluate the metadata associated with each file and/or download the data (courtesy of Data.gov). 


Data.gov 


On October 1, 2011, Data.gov was introduced as “the 
home of the U.S. government’s open data.” With this 
move, U.S. national geospatial assets were brought to- 
gether in one place to theoretically make it easier for 
the public to browse and access over 400,000 maps, da- 
tasets, and geospatial services (Figure A-4). In addi- 
tion, the services, mapping and _ visualization 
capabilities, and data standards behind these sites also 
became accessible. This work was done in coordination 
with a refresh of the Geospatial Platform, which re- 
mains the online home for the Federal Geographic 
Data Committee’s guidance, policies, and standards. 


Data. gov allows the user to search for geospatial infor- 
mation by searching for a particular type of dataset in 
a particular geographic location (Figure A-4). In this 
example, the location was specified as Charleston, SC, 
and the geospatial database was searched to find “Or- 
thoimagery.” The result was a list of 43 orthoimagery 
datasets. This website is ideal for less experienced users 


who are not absolutely sure what image or other geo- 
spatial data they are looking for. 


Commercial Geospatial Data 
Search Engines and/or 
Repositories 


There are numerous commercial image and geospatial 
search engines such as Google earth, bing, ArcGIS On- 
line, and others. Selected characteristics of these three 
geospatial search engines are introduced for demon- 
stration purposes. 


Google, Inc., Google earth Search 
Engine 


The Google earth website provides detailed image and 
other geospatial information about most locations in 
the world. The user can specify the latitude, longitude 
coordinates or interactively locate the area of interest 
on the synthetic Google earth three-dimensional globe. 
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FIGURE A-5 Google earth can be used to look at imagery of almost anywhere on the Earth. In this example, the user re- 
quested geospatial information about the Grand Prismatic Spring in Yellowstone National Park in Wyoming. The historical im- 
age slider can be used to select for display any of the remote sensing imagery stored in the database. In this particular case, a 
DigitalGlobe GeoEye-1 natural-color image of the Grand Prismatic Spring obtained on July 12, 2010, is displayed (user inter- 
face courtesy of Google earth, Inc.; imagery courtesy of DigitalGlobe, Inc.; terrestrial photograph provided by the author). 


Google earth relies heavily on imagery acquired by 
DigitalGlobe, Inc. (e.g., IXONOS, GeoEye, Quick- 
Bird, and WorldView-1 and -2). For example, Figure 
A-5 depicts one image derived for a search on the 
Grand Prismatic Spring in Yellowstone National Park 
in Wyoming. The July 12, 2010, GeoEye-1 natural col- 
or image is draped over U.S. national elevation data 
(NED) yielding a very informative three-dimensional 
display of the spring and surrounding roads and 
mountainous terrain. Google earth also uses public- 
domain imagery such as that provided by the U.S. De- 
partment of Agriculture Farm Service Agency. 


The simple historical image slider user-interface in the 
upper left-hand corner of the interface can be used to 
interactively display each of the dates of imagery of the 
study area that reside in the database. In this particular 
case the following images are available: 


¢ 8/24/1994 USGS Black and White 
¢ 10/23/2003 DigitalGlobe 

¢ 11/6/2004 DigitalGlobe 

* 8/10/2006 DigitalGlobe 


¢ 9/29/2006 USDA Farm Service Agency 
¢ 8/27/2009 USDA Farm Service Agency 
* 7/12/2010 GeoEye-1 


In the Layer sidebar, Google earth provides informa- 
tion about all the DigitalGlobe and SPOT image data 
available for the study area as well as other thematic in- 
formation such as roads that can be overlaid on the im- 
agery. Users often extract screen captures of the 
imagery displayed using Google earth for a variety of 
non-scientific purposes. It is necessary to contact the 
original data provider such as DigitalGlobe, Inc. or 
SPOT Image, Inc. to obtain the original remote sensor 
data. 


Microsoft, Inc., bing Search Engine 


The Microsoft, Inc., bing search engine provides de- 
tailed image information about most locations in the 
world. The user can specify the latitude, longitude co- 
ordinates or interactively locate the area of interest us- 
ing the bing interface. bing relies heavily on digital 
frame camera aerial photography (e.g., UltraCam) ac- 
quired by photogrammetric engineering firms such as 
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FIGURE A-6 The Microsoft Inc., bing interface was used to view imagery of downtown New York City, NY. High spatial reso- 
lution 3Di digital aerial phot ography is displayed. The user has also specified that Bird’s Eye three-dimensional building infor- 
mation available from Pictometry International, Inc. be displayed on top of the remote sensor data (image courtesy of 
Pictometry International, Inc.; user interface courtesy of Bing, Microsoft, Inc.). 


3Di, Sanborn Map Company, Pictometry Internation- 
al, Vexcel Imaging, and others. For example, Figure A- 
6 depicts high spatial resolution natural color digital 
aerial photography of a search centered on downtown 
New York City, NY. Three-dimensional building infor- 
mation provided by Pictometry International’s Bird’s 
Eye view is overlaid on top of the imagery. bing also 
uses public-domain and satellite imagery. Users often 
extract screen captures of bing imagery for a variety of 
non-scientific purposes. It is necessary to contact the 
original data providers to obtain the original remote 
sensor data. 


The Esri, Inc., ArcGIS Online Map and Geoservices 
website offers information associated with basemaps, 
demographic maps, reference maps, specialty maps, 
and imagery (Figure A-7) (http://www.esri.com/soft 
ware/arcgis/arcgis-online-map-and-geoservices/data- 
doors). 


Esri Map Services 

Remote sensing investigations almost always require 
access to other types of geospatial information. Below 
is a list of numerous types of image and map databases 
available through ArcGIS Online Map Services (http:// 
www.esri.com/software/arcgis/arcgisonline/maps 
/maps-and-map-layers). 


World Imagery Basemap: World Imagery provides < 1 
X 1 m satellite or aerial imagery in many parts of the 
world and lower resolution satellite imagery world- 
wide. The map includes NASA Blue Marble: Next 
Generation 500 % 500 m resolution imagery at small 
scales (above 1:1,000,000), i-cubed 15 15 m eSAT im- 
agery at medium-to-large scales (down to 1:70,000) for 
the world, and USGS 15 & 15 m Landsat imagery for 
Antarctica. The map features 0.3 X 0.3 m resolution 
imagery in the continental United States and 0.6 X 0.6 
m resolution imagery in parts of Western Europe from 
DigitalGlobe. In other parts of the world, 1X 1 m reso- 
lution imagery is available from IKONOS, i-cubed Na- 
tionwide Prime, Getmapping, AeroGRID, IGN Spain, 
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FIGURE A-7 The Esri ArcGIS Online Map and Geoservices interface provides access to Map Services (e.g., basemaps, demo- 
graphic, reference maps, and specialty maps) and Image Services (e.g., Landsat GLS-2010, hillshaded basemap, tasseled cap 
imagery, and numerous Landsat band combination datasets). It also provides access to the Esri ChangeMattersfi_ program 
and to MDA NaturalVuefi satellite imagery (screenshots and images courtesy of Esri, Inc.). 


and IGP Portugal. The dataset also contains data from 
the USDA Farm Services Agency National Agriculture 
Imagery Program (NAIP) imagery, USGS Digital Or- 
tho Quarter Quad (DOQQ) imagery, and Aerials Ex- 
press 0.3 to 0.6 m resolution imagery for several 
hundred metropolitan areas. Additionally, imagery at 
different resolutions has been contributed by the GIS 
User Community. 


Other Basemaps: World Street Map, World Topo- 
graphic Map, World Shaded Relief, World Physical 
Map, World Terrain Base, USA Topographic Maps, 
Ocean Basemap, Light Gray Canvas Map, National 
Geographic World Map, and Landsat Hill-shaded 
Basemap are available. 


Demographic Maps: Details about the U.S. population 
are available, including: Average Household Size, Day- 
time Population, Diversity Index, Labor Force Partici- 
pate Rate Median Age, Median Home Value, Median 
Household Income, Median Net Worth, Retail Spend- 
ing Potential, Unemployment Rate, etc. 


Reference Maps: World Boundaries and Places, World 
Boundaries and Places Alternate, World Reference 
Overlay, and World Transportation are available. 


Specialty Maps: DeLorme World Basemap, World 
Navigation Charts, and Soil Survey Maps. 


Esri Image Services 

Esri, Inc., provides a very useful array of image services 
that can be of significant value to people looking for 
various types of remote sensor data. Below is a sum- 
mary of several of the most important image services 
(http://www.esri.com/software/arcgis/arcgis-online- 
map-and-geoservices/image-services). 


Landsat Global Land Survey 2010: The entire Landsat 
image services collection is available through ArcGIS 
Online. This service was compiled from the USGS/ 
NASA Global Land Survey (GLS) 2010 orthorectified 
dataset. The dataset includes imagery from Landsat 5 
TM and Landsat 7 ETM" at 30% 30 m resolution, and 
includes all reflected energy bands, 1.e., Landsat bands 
1, 2,3, 4,5, and 7. The Landsat 7 data are corrected for 
Scan Line Corrector (SLC) errors. They were enhanced 
with radiometric correction and histogram stretching 
to make them more visually appealing. Landsat 8 data 
are now available online (http://www.esri.com/esri- 
news/arcnews/spring 1 4articles/landsat-8-imagery- 
available-for-online-users). 


Landsat Hill-shaded Basemap: Natural color, 15 & 15 
m resolution, pan-sharpened Landsat imagery is or- 
thorectified to 50 m RMSE. It is radiometric corrected 
and enhanced with topographic hill-shading and color 
balancing to produce a basemap that can be overlaid 
with other geospatial information using a GIS. 
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Landsat Color Composites 1975, 1990, 2000, 2005 and 
2010: Esri provides Landsat imagery using the follow- 
ing band combinations (RGB = bands x, y, and z): 


¢ False-color/Near-infrared (4,3,2), 

¢ Agriculture (5,4,1), 

e Atmospheric Penetration (7,5,4), 

¢ Healthy Vegetation (4,5,1), 

¢ Land/Water Boundary (4,5,3), 

¢ Natural with Atmospheric Penetration (7,4,2), 
¢ Shortwave Infrared (7,4,3), 

¢ True Color/Natural Color (3,2,1), and 

¢ Vegetation Analysis (4,5,3). 


Tasselled Cap Service: This image service is based on 
USGS GLS 2010 orthorectified Landsat data and the 
tasseled cap transformation discussed in Chapter 7. 
Tasseled cap data are useful for monitoring vegetation 
development over a growing season for predicting crop 
maturity and yield. The Brightness image contains 
most of the variability in the image and is similar to 
panchromatic imagery. The Greenness image is useful 
for discriminating between plant species and quantify- 
ing plant vigor and biomass. The Wetness image is re- 
lated to plant and soil moisture characteristics. The 
remaining bands contain noise such as haze, clouds 
and atmospheric affects. Brightness, Greenness and 
Wetness images are displayed as a color composite 
(RGB) where bright human-made and natural areas 
are red, vegetation is in shades of green, and wet or 
moist features are blue. 


Landsat Change Mattersfi: This online service mosaics 
multiple-date Landsat images irrespective of the Land- 
sat sensor that acquired them (Green, 2011). Change- 
Mattersfi displays user-specified individual dates of 
imagery using standard band combinations, including: 
near-infrared color composites (e.g., TM bands 4,3,2) 
useful for vegetation studies and urban analysis; natu- 
ral color with atmospheric penetration (e.g., TM bands 
7,4,2) best suited for analysis of vegetation and some 
urban studies; healthy vegetation; land/water bound- 
ary detection; and vegetation analysis providing infor- 
mation for agriculture and forest management. The 
NDVI of the two images is computed on the fly and 
used to document the change in NDVI between the 
two images (http://www.esri.com/software/landsat- 
imagery/viewer). Two examples of the Change Mat- 
tersfi user interface were presented in Chapter 12. 


Esri also provides Landsat NDVI (Normalized Differ- 
ence Vegetation Index) Change information associated 
with the following time periods: 1975 to 1990; 1975 to 
2000; 1975 to 2005; 1990 to 2000; and 2000 to 2005. 


MDA NaturalVue Satellite Imageryfi: This dataset 
presents NaturalVuefi 15 X 15 m satellite imagery of 
the world created by MDA Information Systems LLC. 
NaturalVuefi is a natural color worldwide seamless 
mosaic derived over 8,600 Landsat-7 multispectral im- 
ages. This single global dataset has a positional accura- 
cy of better than 50 m RMSE. 


{4 Digital Elevation Data 


Accurate digital elevation information is very impor- 
tant for many remote sensing and GIScience applica- 
tions. A large number of commercial photogrammetric 
engineering firms and surveying/mapping companies 
provide access to high-quality digital-elevation infor- 
mation for a fee. 


* DRG—Digital Raster Graphics (USGS at www. 
libremap.org). 


Sometimes it is useful to analyze historical topographic 
information. Fortunately, the hard-copy USGS topo- 
graphic maps created over many decades have been 
scanned to create digital raster graphics (DRGs) from 
the 1:24,000, 1:25,000, 1:63,360 (Alaska), 1:100,000, 
and 1:250,000-scale topographic map series. Coverage 
includes the standard USGS series quadrangle maps of 
the U.S. and its trusts and territories. Information in- 
side the map neatlines is georeferenced to the surface 
of the Earth and fit to the Universal Transverse Merca- 
tor map projection. The horizontal positional accuracy 
and datum of the DRG matches the accuracy and da- 
tum of the source map. The maps were scanned at a 
minimum resolution of 250 dpi. Historical DRGs may 
be downloaded free from the U.S. Geological Survey at 
http://libremap.org/. 


Several other important sources of digital elevation in- 
formation are presented in Table A-1, including: 


¢ GTOPO30—World digital elevation model (USGS) 
« NED—National Elevation Dataset (USGS) 

¢ Topographic and Bathymetric (USGS, NOAA) 

¢ Topographic Change Information (USGS) 


¢ SRTM—Shuttle RADAR Topography Mission 
(NASA JPL) 

¢ ASTER Global Digital Elevation Map 
(METI and NASA JPL), and 


¢ NEXTMap World 30 Digital Surface Model 
(Intermap, Inc.). 
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TABLE A-1 Digital Elevation Data. 


Data 


Description 


GTOPO30 


GTOPO30 is a global digital elevation model with 
a horizontal grid spacing of 30 arc-seconds (ap- 
proximately 1% 1 km). It was derived from numer- 
ous sources of topographic information. It has 
been used extensively for regional, continental, 
and global studies and for numerous global 
change applications. Compare this GTOPO30 
DEM of Mt. Kilimanjaro with the SRTM DEM to 
appreciate the coarse resolution of the GTOPO30 
data. Source: http://eros.usgs.gov/#/Find_Data/ 
Products_and_Data_Available/gtopo30_info. 


Mt. Kilimanjaro 


NED— 
National 
Elevation 
Dataset 


The National Elevation Dataset is the 30% 30 m 
resolution elevation layer of The National Map. 
NED has evolved into a multi-resolution dataset 
that provides the best available elevation data, in- 
cluding 3X 3 m or better LIDAR-derived elevation 
data. Source: http://ned.usgs.gov. See Gesch 
(2007) for a detailed description of the National 
Elevation Dataset characteristics. 


Topographic- 
Bathymetric 
Information 


Topographic-bathymetric (topobathy) data are 
merged topography (land elevation) and bathym- 
etry (water depth) data in a single product useful 
for inundation mapping and other applications. 
Topography data come from the National Eleva- 
tion Dataset (NED). Bathymetry data are provided 
by the NOAA GEOphysical DAta System (GEO- 
DAS) .The example depi cts top obathy data of a 
part of the Puget Sound, WA. Source: http:// 
topotools.cr.usgs.gov/topobathy_viewer/. 


Topographic 
Change 
Information 


The need for information on the extent of human 
geomorphic activity resulted in the creation of the 
first-ever accounting of topographic change 
across the U.S. The primary types of topographic 
changes resulting from human geomorphic activi- 
ty include surface mining, road construction, ur- 
ban development, dam construction, and landfills. 
The example shows: (left) NED DEM of a gold 
mine in Carlin, NV; (right) cut (blue) and fill (red) 
areas identified using SRTM-derived DEM. 
Source: http://topochange.cr.usgs.gov 
/topochange_viewer/viewer.htm. 


SRTM— 
Shuttle 
RADAR 
Topography 
Mission 


SRTM data were obtained by the Space Shuttle 
Endeavour in 2000. SRTM acquired data for over 
80 percent of the Earth's land surface between 60 
degrees N and 56 degrees S latitude. SRTM imag- 
es are available at no charge as FTP downloads. 
Compare the GTOPO30 DEM of Mt. Kilimanjaro 
with the SRTM DEM to appreciate the fine spatial 
resolution of the SRTM-derived DEM. Source: 
http://srtm.usgs.gov/index.php. 


Mt. Kilimanjaro 
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TABLE A-1 Digital Elevation Data (continued). 


Data 


Description 


Example 


ASTER 

Global Digital 
Elevation 
Model 
(GDEM V2) 


The Japanese Ministry of Economy, Trade, and In- 
dustry (METI) and NASA released the Advanced 
Spaceborne Thermal Emission and Reflection Ra- 
diometer (ASTER) Global Digital Elevation Model 
Version 2 (GDEM V2) on October 17, 2011. ASTER 
GDEM coverage is from 83 degrees N to 83 de- 
grees S latitude, encompassing 99 percent of 
Earth's landmass. The improved GDEM V2 adds 
260,000 additional stereo-pairs, improving cover- 
age and reducing the occurrence of artifacts. The 
ASTER GDEM V2 is in GeoTIFF format with 30-m 
postings and 1 1 degree tiles. ASTER GDEM V2 
data are available free to users worldwide from 
the Land Processes Distributed Active Archive 
Center (LP DAAC) and J-spacesystems. The 
GDEM V2 coverage is shown. Source: http://aster- 
web. jpl.nasa.gov/gdem.asp. 


NEXTMap 
World 30 DSM 
(Intermap, 
Inc.) 


Intermap Technologies, Inc., NEXTMap World 30 
DSM is a combination of 90-m Shuttle Radar Top- 
ographic Mission (SRTM) v2.1 data, 30-m ASTER 
Global DEM v2.0, and 1-km GTOPO30 data which 
has been ground controlled using LIDAR data 
from NASA's Ice, Cloud and Land Elevation Satel- 
lite (ICESat) collection, resulting in a 25-cm RMSE 
dataset for vertical control of the DSM. The result- 


http://www.intermap.com/en-us 
/databases/world30.aspx 


entire land mass of the planet. 


ing product is a 30-m GSD DSM that covers the 


Hydrography Data 


Hydrologic information is critical to the successful 
modeling of many natural (e.g., streams) and human- 
made (e.g., canals) geospatial relationships. Table A-2 
summarizes the availability of detailed hydrologic in- 
formation from: 


¢ NHD—National Hydrologic Database (USGS) 


* EDNA—Elevation Derivatives for National Appli- 
cations (USGS). 


Land Use/Land Cover and 
Bio diversity/Habitat Data 


Land use is the human use of the terrain. Land cover is 
the biophysical material present on the surface of the 
Earth such as vegetation, soil, water, rocks, etc. The 
land cover is the habitat for animals, birds, and aquatic 
species. Several of the more important land use/land 


cover and biodiversity/habitat resources are presented 
in Table A-3, including: 


« NLCD—National Land Cover Database (USGS) 


« C-CAP—Coastal Change Analysis Program 
(NOAA) 


¢ GAP Analysis Program (USGS) 
¢ NWI—National Wetlands Inventory (USFWS) 


In addition, there is the National Phenology Network. 
Phenology is the study of nature’s calendar. For exam- 
ple, the network contains information about when 
peach trees blossom, when alligators build their nests, 
when salmon swim upstream to spawn, or when corn is 
fully ripe. The National Coordinating Office of the 
Network is a resource center that facilitates and en- 
courages widespread collection, integration, and shar- 
ing of phenology data and related information. Source: 
USA National Phenology Network, 1955 East 6th 
Street, Tucson, AZ 85721. 
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TABLE A-2 Hydrologic Information. 


Data 


Description 


Example 


NHD— 
National 
Hydrography 
Dataset 


The NHD is the surface water component of The 
National Map. The NHD is a digital vector dataset 
easily used in GIS hydrologic analysis. It contains 
features such as lakes, ponds, streams, rivers, ca- 
nals, dams, and stream gages. These data are de- 
signed to be used in general mapping and in the 
analysis of surface-water systems. The NHD con- 
tains vector-surface-water-hydrography informa- 
tion obtained from topographic maps and 
additional sources. It is available nationwide as 
medium resolution at 1:100,000-scale and as high 
resolution at 1:24,000-scale or better. In Alaska, 
the NHD is available at 1:63,360-scale. A few “lo- 
cal resolution” areas are available at varying 
scales. The hydrography of the U.S. is organized 
by drainage areas. The sub-basin 8-digit Hydro- 
logic Unit Code (HUC) drainage area is the most 
practical area for high resolution NHD. Subre- 
gions (4-digit HUCs) are composed of varying 
numbers of subbasins. The NHD is available in an 
Esri personal geodatabase format known as 
NHDinGEO, a file-based geodatabase format, 
and in Esri shapefile format known as NHDGEOin- 
Shape. The NHD is organized by hydrologic units, 
but can be downloaded in various geographic ex- 
tents. This map displays NHD data of Hobble 
Creek, UT. Source: http://nhd.usgs.gov/. 


EDNA— 
Elevation 
Derivatives for 
National 
Applications 


EDNA is a multi-layered database derived from 
the NED, which has been hydrologically condi- 
tioned for improved hydrologic flow modeling. 
The seamless EDNA database provides 30 30m 
raster and vector data layers including: 


Aspect 

Contours 

Filled DEM 

Flow Accumulation 
Flow Direction 

Reach Catchment Seedpoints 
Reach Catchments 
Shaded Relief 

Sinks 

Slope 

Synthetic Streamlines 


Hydrologically conditioned elevation data, pro- 
cessed to create hydrologic derivatives, can be 
useful in many topologically based hydrologic vi- 
sualization and investigative applications. Drain- 
age areas upstream or downstream from any 
location can be accurately traced, facilitating 
flood analysis investigations, pollution studies, 
and hydroelectric power generation projects. This 
is a map of reach catchment seedpoints. Source: 
http://edna.usgs.gov/. 
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TABLE A-3 Land Use/Land Cover and Biodiversity/Habitat Information. 


Data Description Example 
NLCD— The Multi-Resolution Land Characteristics (MRLC) mm Open Water 
National Lan Consortium developed the NLCD, which is a 16 to tae Low-Int. Resident 
ational Land 21 class land cover dataset of the U.S. derived ean Wigh-tor, Residene 
Cover from 30% 30 m Landsat TM data. The NLCD state 0 Cn a 
Database datasets are extracted from regional datasets that ; ~ Quarry Strip Mine 
1992, 2001 are mosaics of Landsat TM scenes. NLCD 1992 is mm Transitional 
: 7 a land-cover dataset. NLCD 2001 is a land-cover fm Deciduous Forest 
2006, 2011 database composed of three elements: land cov- ta Evorgroen Forest 
er, impervious surface, and canopy density. NLCD poy AOS 
2006 and 2011 are now available to the public. mma 
The land cover in Washington, DC, in 1992 is pan IE: vy 
shown. Source: http://www.mrlc.gov/. wa Small Grains 
Other Grasses 
C-CAP— C-CAP is a database of land cover and land cover 
Coastal change for the coastal regions of the U.S. pro- 
Ch duced every 5 years. C-CAP provides inventories 
ange of coastal intertidal areas, wetlands, and adjacent 
Analysis uplands. The products consist of land cover maps 


for each date of analysis, as well as a file that high- 
lights what changes have occurred between these 
dates. NOAA also produces high resolution C- 
CAP land cover products, for select geographies. 
C-CAP provides the “coastal expression” of the 
NLCD, and contributes to the Earth Cover layer of 
the National Spatial Data Infrastructure. The ex- 
ample shows C-CAP data of Pearl Harbor, HI. 
Source: http://coast.noaa.gov/digitalcoast/data 
/ccapregional/?redirect=301ocm. 


Program 


GAP Analysis The GAP Analysis Program is part of the National 
Prog ram Biological Information Infrastructure (NBII). The 

goal is to keep common species common by pro- 
tecting them before they become threatened. 
GAP activities focus on the creation of state and 
regional databases and maps that depict patterns 
of land management, land cover, and biodiversity. 
These data can be used to identify “gaps” in con- 
servation where an animal or plant community is 
not adequately represented on the existing net- 
work of conservation lands. The example depicts 
the U.S. GAP land cover map which contains 551 
ecological systems and modified ecological sys- 
tems. Source: http://gapanalysis.usgs.gov/. 


NWI— The U.S. Fish & Wildlife Service's National Wet- 
National lands Inventory provides current geospatial data 
on the status, extent, characteristics, and func- 
Wetlands tions of wetland, riparian, deepwater, and related 
Inventory aquatic habitats in priority areas to promote the 


understanding and conservation of these resourc- 
es. The NWI serves on-line map information for 
82% of the conterminous United States, 31% of 
Alaska, and 100% of Hawaii. The goal is to com- 
plete and maintain a seamless digital wetlands da- 
taset for the entire nation that will become the 
Wetlands Data Layer of the NSDI. The NWI exam- 
ple is of Crayvik, FL. Source: http://www.fws.gov 
/wetlands/. 
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Road Network and 
Population Demographic 
Data 


Topologically correct road-network information is in- 
dispensable for navigation, business decision making 
and for use in various remote sensing investigations. 
The U.S. Census Bureau provides the MAF/TIGER— 
Master Address File/Topologically Integrated Geo- 
graphic Encoding and Reference System free of charge 
to the public. Details are provided in Table A-4. 


The Census Bureau also provides population demo- 
graphic information aggregated to various spatial enu- 
meration districts (Table A-4). 


LandScan, created by the Oak Ridge National Labora- 
tory, predicts the spatial distribution of people at dif- 
ferent times of day anywhere in the world at a spatial 
resolution of 1X 1 km (Table A-4). 


{4 Remote Sensor Data—-Pu blic 


Remote sensing data are available from orbital and 
aerial platforms and sensors. Table A-5 provides infor- 
mation about some of the public remote sensing sys- 
tems commonly used to collect data. 


e ALI—Advanced Land Imager (NASA) 
¢ ASTER—Advanced Spaceborne Thermal Emission 
and Reflection Radiometer (NASA) 


« AVHRR—Advanced Very High Resolution Radi- 
ometer (NOAA) 


« AVIRIS—Advanced Visible Infrared Imaging Spec- 
trometer (NASA) 


¢ Declassified Satellite Imagery (USGS) 

« DOQ—Digital Orthophoto Quadrangles (USGS) 

¢ Hyperion—Hyperspectral Imager (NASA) 

¢ Landsat Multispectral Scanning System (MSS), 
Thematic Mapper (TM), Enhanced Thematic Map- 
per Plus (ETM*), and Landsat 8 (NASA/USGS) 

« LiDAR—Light Detection and Ranging (USGS) 

« MODIS—Mocderate Resolution Imaging Spectrom- 
eter (NASA) 

« NAIP—National Agriculture Imagery Program 
(USDA) 


¢ Suomi—NPOESS Preparatory Project 
(NOAA). 


(NPP) 


Remote Sensor Data— 
Commercial and 
International 


Remote sensor data available from selected commercial 
and international vendors are provided in Table A-6. 


¢ CASI-1500, SASI-600, MASI-600 and TASI-600 
(ITRES, Inc.) 

¢ EROS A and EROS B (ImageSat Intl., Inc.) 

¢ GeoEye-1 and GeoEye-2 (DigitalGlobe, Inc.) 

¢ HyMap Hyperspectral imagery (HyVista, Inc.) 

« IKONOS-2 (DigitalGlobe, Inc.) 


¢ Indian IRS A-D, CartoSat 1-3, ResourceSat 1-2 
(www.isro. gov.in) 


¢ Korean KOMPSAT 1-5 (http://www.kari.re.kr) 

¢ Pictometry (EagleView Technologies, Inc.) 

¢ Pleiades-1 and Pleiades-2 (www.astrium-geo.com) 
* QuickBird (DigitalGlobe, Inc.) 

* RapidEye (RapidEye, Inc.) 

¢ Sentinel-2 (European Space Agency) 

¢ SPOT 1-6 (www.astrium-geo.com) 


¢ WorldView-1, WorldView-2, World View-3 
(Digital Globe, Inc.). 
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TABLE A-4 Road Network and Population Demographic Information. 


Data Description Example 


MAF/TIGER/ TIGER/Linefi Shapefiles are spatial data extracts 
Line from the U.S. Census Bureau MAF/TIGER data- 
base (Master Address File/Topologically Integrat- 
ed Geographic Encoding and Reference System), 
containing features such as topologically correct 
roads, railroads, and rivers, as well as legal and 
statistical geographic areas. They are available to 
the public at no charge and are typically used to 
provide the digital base map for GIS software.The 
TIGER/Linefi Shapefiles do not include demo- 
graphic data, but they contain geographic entity 
codes that can be linked to the Bureau's demo- 
graphic data, available on American FactFinder 
described below. The TIGER /Line Shapefiles are 
provided in four types of coverages: 


County-based 

State-based 

Nation-based 

American Indian Area-based 


Source: http://www.census.gov/geo/maps-data 
/data/tiger.html. 


2010 Census The 2010 Census reported that 308.7 million people live in the U.S., a 9.7 percent increase from 
Population the 2000 Census population of 281.4 million. Below are some of the most important 2010 Unit- 
Demog raphics ed States Census products (tabular and feature-based). 


American FactFinder (AFF) 

The AFF is a data access system that will find and retrieve geographic information and plot the 
information on an interactive map. Source: http://factfinder2.census.gov/faces/nav/jst/pages 
/index.xhtml. 


Cartographic Boundary Files 
Generalized, digital files suitable for use with GIS in small-scale thematic mapping. Source: 
http://www.census.gov/geo/maps-data/data/tiger-cart-boundary.html. 


Census 2010 County Block Maps 
Color maps, showing census blocks, voting districts, and other feature details. Source: http:// 
www.census.gov/geo/maps-data/maps/block/2010/. 


Voting District/State Legislative District Reference Maps 
Color maps, showing voting districts and/or state legislative districts. Source: http://www 
.census.gov/geo/maps-data/maps/reference-sld. html. 


LandScan LandScan was developed by Oak Ridge National Laboratory (ORNL) through funding by the 
Population DOD and the DOE. At approximately 14 1 km, LandScan is the highest resolution global pop- 

nila 2 ulation distribution data available and represents an ambient population (average over 24 
Distribution hours). The LandScan algorithm uses spatial data and image analysis technologies and a multi- 
Modeling variable dasymetric modeling approach to disaggregate census counts within an administrative 
boundary. LandScan population distribution models are tailored to match the data conditions 
and geographical nature of individual countries and regions. The LandScan dataset is available 
free to U.S. Government agencies. Others, such as educational institutions or research scien- 
tists, must register with LandScan to determine license fees (if any) for obtaining LandScan re- 
sources. The data are distributed in both an ESRI grid format and an ESRI binary raster format. 
The dataset has 20,880 rows and 43,200 columns covering north 84 degrees to south 90 de- 
grees and west 180 degrees to east 180 degrees. Source: http://www.ornl.gov/sci/landscan 
/index.shtml. 
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TABLE A-5 Remote Sensor Data—Public. 


Data 


Description 


Example 


ASTER— 
Advanced 
Spaceborne 
Thermal 
Emission and 
Reflection 
Radiometer 


ASTER is carried onboard NASA's Terra satellite. 
ASTER began data collection in 2000 and consists 
of three sensors that obtain images in multiple 
spatial resolutions (154 15, 30 30, and 90H 90 
m). ASTER is the only high-resolution imaging sen- 
sor on Terra. The primary goal of the ASTER mis- 
sion is to obtain high-resolution image data in 14 
channels (including thermal infrared) over the en- 
tire land surface, as well as black and white stereo 
images. With a revisit time of between 4 and 16 
days, ASTER provides repeat coverage of chang- 
ing areas on Earth’s surface. This is an ASTER im- 
age of the Patagonia Glacier in Chile obtained on 
May 2, 2000. Source: http://asterweb.jpl.nasa.gov 
/gdem.asp. 


AVHRR— 
Advanced 
Very High 
Resolution 
Radiometer 


The Advanced Very High Resolution Radiometer 
(AVHRR) is an optical multispectral scanner flown 
aboard NOAA orbiting satellites. The instrument 
measures reflected sunlight and emitted radiation 
(heat) from Earth in the visible (Channel 1), near- 
infrared (Channel 2), and thermal infrared (Chan- 
nels 3, 4, and 5) regions of the electromagnetic 
spectrum. There is fairly continuous global cover- 
age since June 1979, with morning and afternoon 
acquisitions available. The resolution is 1.14 1.1 
km at nadir. This is an image of the Gulf Stream off 
of North and South Carolina. Source: http:// 
eros.usgs.gov/#/Find_Data/Products_and_Data_ 
Available/AVHRR. 


AVIRIS— 
Airborne/ 
Visible Imaging 
Spectrometer 


AVIRIS is a unique optical sensor that delivers cali- 
brated images of the upwel ling spectral radiance 
in 224 contiguous spectral channels (bands) with 
wavelengths from 400 to 2,500 nanometers. 

The main objective of the AVIRIS remote sensing 
system is to identify, measure, and monitor con- 
stituents of the Earth's surface and atmosphere 
based on molecular absorption and particle scat- 
tering signatures. Research with AVIRIS data is 
predominantly focused on understanding pro- 
cesses related to the global environment and cli- 
mate change. This is an AVIRIS image of the lower 
portion of the San Francisco Bay, California. 
Source: http://aviris.jpl.nasa.gov/. 


Declassified 
Satellite 
Imagery 


Almost 90,000 declassified satellite images are 
maintained in the USGS EROS declassified satel- 
lite image archive. The images were captured by a 
variety of intelligence satellites, including CORO- 
NA, ARGON, LANYARD, KH-7, and KH-9 between 
1960 and 1980. Coverage is global, but the geo- 
graphic distribution is uneven. Some images are 
high resolution (although not georeferenced). This 
is an image of the western edge of Dakar, Sene- 
gal, Africa, acquired by the KH-7 satellite in 1966. 
Source: http://eros.usgs.gov/#/Find_Data 
/Products_and_Data_Available 
/Declassified_Satellite_Imagery_-_2. 
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TABLE A-5 Remote Sensor Data—Public (continued). 


Data Description Example 
DOO—Digital A Digital Orthophoto Quadrangle (DOQ) is a 
Orthophoto computer-generated image of an aerial photo- 


Quadrangles 


graph in which the image displacement caused by 
terrain relief and camera tilt has been removed. 
The DOO combines the image characteristics of 
the original photograph with the georeferenced 
qualities of a map. DOQs can be black-and-white 
(B/W), natural color, or color-infrared (CIR) images 
with 1% 1m ground resolution. They cover an 
area measuring 3.75-minutes longitude by 3.75- 
minutes latitude or 7.5-minutes longitude by 7.5- 
minutes latitude. A CIR DOO of downtown Wash- 
ington, DC, is shown. Source: https://Ita.cr.usgs 
.gov/DOOs. 


Landsat— 
MSS, TM 
ETM*, 
Landsat 8 


The first two Earth Resources Technology Satellite 
(ERTS) were launched in 1972 and 1975. They 
were subsequently renamed Landsat satellites. 
Additional Landsat satellites were launched in 
1978, 1982, 1984, 1993 (did not achieve orbit), 
1999 and 2013. The Landsat satellites have carried 
a variety of sensors, including the Multispectral 
Scanning System (MSS; 80 8 80 m), Thematic 
Mapper (TM; 30 and 60 m), and Enhanced The- 
matic Mapper (ETM*; 15, 30 and 60 m). Landsat 8 
was launched on February 11, 2013, with its Oper- 
ational Land Imager; 15 and 30 m) and Thermal In- 
frared Sensors (100 m). All Landsat data acquired 
from 1972 are available at no charge from the 
USGS. A Landsat 7 image draped over an SRTM 
digital elevation model of southern Malawi, Africa, 
near the Mozambique border is shown. Source: 
http://landsat.gsfc.nasa.gov/images/archive 
/f0005.html. 


LiDAR— Light 
Detection and 
Ranging 


LiDAR data collection is one of the most impor- 
tant sources of digital surface models that provide 
detailed information about the x,y, and z-location 
of masspoints for all features on the surface of the 
Earth. With accurate processing, the masspoints 
can be edited to produce bare-earth digital ter- 
rain models. Additional processing can yield bio- 
physical information about the vegetation. LIDAR 
can also be used to obtain bathymetric informa- 
tion in shallow clear water. Click is the USGS Cen- 
ter for LIDAR Information Coordination and 
Knowledge at http://lidar.cr.usgs.gov/. 


MODIS— 
Moderate 
Resolution 
Imaging 
Spectrometer 


MODIS is flown on both the NASA Aqua and Terra 
satellites. It collects data in 36 spectral bands at 
three spatial resolutions: 250 X 250, 500 X 500, 
and 1,000 X 1,000 m. MODIS collects imagery of 
the entire Earth every 1 to 2 days. The data are 
excellent for regional, continental, and global 
analyses of relatively small-scale problems such as 
dynamic land, ocean, ice, and atmospheric pro- 
cesses and global environmental change. 

Source: http://modis.gsfc.nasa.gov/gallery/#. 
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TABLE A-5 Remote Sensor Data—Public (continued). 


Microwave Sounder (ATMS); Cross-track Infrared 
Sounder (CrlS); Ozone Mapping and Profiler suite 
(OMPS); and Clouds and the Earth's radiant Ener- 
gy System (CRES) radiometer. This image of the 
continental U.S. at night is a composite assembled 
from data acquired by the Suomi NPP satellite in 
April and October 2012. The image was made 
possible by the “day-night band” of the VIIRS, 
which detects light in a range of wavelengths from 
green to near-infrared and uses filtering tech- 
niques to observe dim signals such as city lights, 
gas flares, auroras, wildfires and reflected moon- 
light. Credit: NASA Earth Observatory/NOAA 
NGDC. Source: http://npp.gsfc.nasa.gov/. 


Data Description Example 
NAIP— The NAIP acquires imagery during the agricultural 
National peak growing seasons for the conterminous Unit- 

F ed States. The “leaf-on” orthophoto images are at 
Agriculture 1X 1 mor 2M 2m spatial resolutions. The 1K 1m 
Imagery imagery provides updated digital orthophoto im- 
Prog ram agery. The 2 2 m imagery supports the USDA 

Farm Services programs that require current imag- 
ery acquired during the agricultural growing sea- 
son but do not require a higher resolution. 
Included in this imagery category is 1K 1 m data 
acquired through certain state programs as “leaf- 
off.” Source: http://www. fsa.usda.gov/FSA 
/apfoapp?area=home&subject=prog&topic=nai. 
Suomi— The Suomi NPOESS Preparatory Project (NPP) 
NPOESS satellite was launched on October 28, 2011. It has 
five imaging systems, including: Visible Infrared 
Preparatory Radiometer Suite (VIIRS) consisting of 22 channels 
Project between 0.4 and 12 Km; Advanced Technology 


TABLE A-6 Remote Sensor Data—Commercial and International. 


Data 


Description 


Source 


CASI-1500 
SASI-600 
MASI-600 
TASI-600 


ITRES, Inc. of Canada markets several airborne 
optical hyperspectral remote sensing systems: 
[CASI-1500 (0.37 - 1.05 im) and SASI-600 
(0.95 — 2.45 Xm)], and hyperspectral thermal 
infrared systems: [MASI-600 (4 — 5 Xm) and 
TASI-600 (8 — 11.5 Xm)]. Typical ground spatial 
resolution is between 1 and 10 m. 


www.itres.com 


EROS A 
EROS B 


EROS A was launched on December 5, 2000, 
with a single 1.9 1.9 m panchromatic band 

(0.5 — 0.9 Xm). EROS B was launched on April 25, 
2006, with the identical sensor. 


www.imagesatintl.com 
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TABLE A-6 Remote Sensor Data—Commercial and International (continued). 


Data Description Source 
GeoEye-1 GeoEye-1 was launched on September 6, 2008, www.digitalglobe.com 
GeoEve-2 with a 0.41 & 0.41 m panchromatic band and four 

7 1.65 1.65 m multispectral bands. GeoEye-2 has 


a 0.25 M 0.25 m panchromatic band and four 
1.65 1.65 m multispectral bands. GeoEye, Inc. 
merged with DigitalGlobe, Inc. in February, 2013. 


HyMap Airborne HyMap hyperspectral data are recorded | www.hyvista.com 
in 128 bands in the region from 450 — 2,480 nm. 
Typical ground resolution is between 2 and 10 m. 


IKONOS-2 IKONOS-2 was launched on April 27, 1999. It has www.digitalglobe.com 
a 0.82 X 0.82 m panchromatic band and four 
3.2 3.2 m multispectral bands. GeoEye, Inc. 
merged with DigitalGlobe, Inc. in February, 2013. 


Indian The Indian Space Research Organization (ISRO) www.isro.gov.in 
IRS-1A launched IRS-1A on March 17,1988; IRS-1B on 

August 29, 1991; IRS-1C on December 28, 1995; 
IRS-1B and IRS-1D on September 29, 1997. CartoSat-1 
IRS-1C was launched on May 5, 2005, with a 2.5% 2.5m 
IRS-1D panchromatic band on two cameras. CartoSat-1A 

will have 1.25 1.25 m pan band, 2.5K 2.5m 

iz multispectral bands, an X m erspectra 

CartoSat-1 Itispectral bands, and 30 30 m hyperspectral 
CartoSat-2 bands. CartoSat-2 was launched on January 10, 
CartoSat-2A 2007, with a 1% 1 m pan band. CartoSat-2A was 


launched on April 28, 2008, with the same specifi- 
Cartosat-2B cations. CartoSat-2B was launched on July 12, 
Cartosat-3 2010, with a 1 1 m pan band. CartoSat-3 will 
ResourceSat-1 a : ene pe ae meta ae Cate was 
aunched on October 17, , Wi ree sensors 
ResourceSat-2 recording imagery at 5.8% 5.8 m, 23.5 23.5 m, 
and 56 & 56 m spatial resolutions. ResourceSat-2 
was launched on April 20, 2011, with swath width 


improvements. 
Korean hid akan es — oe ee http://www.kari.re.kr (Korean) 
aunche -1 on December 21, ; 
KOMPSAT-1 with 6& 6 m panchromatic band. It was retired in 
KOMPSAT-2 2008. KOMPSAT-2 was launched on July 27, 2006, 
KOMPSAT-3 and has a 1 1m panchromatic band and four 4 
= X 4m multispectral bands. KOMPSAT-3 was 
KOMPSAT-4 launched on May 18, 2012, with a 0.8% 0.8m 
KOMPSAT-5 panchromatic band anda 4K 4m VNIR multispec- 


tral sensor. KOMPSAT-3A will have sensors similar 
to KOMPSAT-3 except for a 5.5 X 5.5 m thermal 
infrared band in the region 3 - 6 Xm. KOMPSAT-5 
is to have a Synthetic Aperture Radar (SAR) that 
can acquire data at 1 1 m spatial resolution. 


PICTOMETRY Pictometry International and EagleView Technolo- | www.eagleview.com/Products 
gies merged in 2013. Pictometry imagery consists | /lmageSolutionsAnalytics 

of a single Nadir (vertical) image and four low- /Pictometrylmagery.aspx 
oblique images (North, East, South, and West) 
collected at each exposure station. Natural color 
and color-infrared imagery can be collected. The 
flightlines overlap by 20% to 30% resulting in a 
great diversity of views of each terrain object. 
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TABLE A-6 Remote Sensor Data—Commercial and International (continued). 


2012. Each satellite sensor has a0.5H 0.5m 
panchromatic band and four multispectral bands 
at 2 2 m spatial resolution. 


Data Description Source 
Pleiades-1 Pleiades-1 was launched on December 17, 2011, www.astrium-geo.com/pleiades/ 
Pleiades-2 and Pleiades-2 was launched on December 2, 


QuickBird 


DigitalGlobe, Inc. launched QuickBird on October 
18, 2001, with a 0.65 XH 0.65 m panchromatic band 
and four 2.62 X 2.62 m multispectral bands. 


www.digitalglobe.com 


RapidEye 


Five identical RapidE ye satellites were launched 
on August 29, 2008. Each RapidEye sensor system 
has five multispectral bands at 6.5 X 6.5 m spatial 
resolution (5 X 5 m when processed to become 
orthoimagery). 


www.rapideye.com 


Sentinel-2 


The European Space Agency (ESA) is developing 
a new family of missions called Sentinels specifi- 
cally for the operational needs of the Copernicus 
programme. Sentinel-2 will have a Multi-Spectral 
Instrument (MSI) with a total of 13 bands: 4 VNIR 
bands (400 — 750 nm) at 10K 10 m, 6 SWIR bands 
(1,300 — 3,000 nm) at 20 X 20 mm and 3 bands at 
60 m spatial resolution dedicated to atmospheric 
correction and cloud screening. 


www.esa.int/Our_Activities/Observing_ 
the_Earth/Copernicus/Sentinel-2 


SPOT 1 
SPOT 2 
SPOT 3 
SPOT 4 
SPOT 5 
SPOT 6 
SPOT 7 


SPOT 1, 2, and 3 are retired and SPOT 4, 5, and 6 
are operational. Archived SPOT 1, 2, and 3 data 
have a 10 10 m panchromatic band and three 
20 X 20 m multispectral bands. SPOT 4 has a 
10H 10 m panchromatic band, three 20 X 20 m 
multispectral bands, and one 20 HX 20 m SWIR 
band. SPOT 5 has a 2.5 8 2.5 m panchromatic 
band, three 10 10 m multispectral bands and 
one 20% 20 m SWIR band. SPOT 4 and SPOT 5 
have a unique 1.15 1.15 km Vegetation sensor 
w th four multispectral bands. SPOT 6 was 
launched on September 9, 2012, with a 1.5 X 

1.5 m panchromatic band and four multispectral 
bands at 6X 6m_ spatial resolution. SPOT 7 was 
launched on June 30, 2014, with the same pay- 
load as SPOT 6. 


www.astrium-geo.com/en/147-spot-6-7- 
satellite-imagery 


WorldView-1 
WorldView-2 
WorldView-3 


WorldView-1 was launched on September 18, 
2007, with a 0.5 8 0.5 m panchromatic band. 
WorldView-2 was launched October 8, 2009, with 
a 0.46 X 0.46 m panchromatic band and eight 
multispectral bands at 1.85 X 1.85 m spatial reso- 
lution. WorldView-3 was launched on August 13, 
2014, with a 0.31 X 0.31 m panchromatic band, 
eight 1.24 1.24 m multispectral bands, and 
eight 3.7 X 3.7 m SWIR bands. In June, 2014, Dig- 
italGlobe received permission from the U.S. De- 
partment of Commerce to sell WorldView-3 
imagery at their best available resolution: 0.25 
0.25 m panchromatic and 1% 1m multispectral. 


www.digitalglobe.com 


INDEX 


Numbers 

2D, evaluation of pixel brightness 
values, 138 

3D, evaluation of pixel brightness 
values, 138 

8-bit color processing, 161-162 

24-bit color processing, 164 


A 
Absolute atmospheric correction 
of attenuation, 212-214 
band-by-band spectral polishing, 
470-471 
correction based on radiative 
transfer model, 216-220 
correction using ELC, 220-223, 
471 
diffuse sky irradiance in, 216 
Radiative Transfer-based 
Absolute Atmospheric 
Correction, 469-471 
target and path radiance in, 
214-215 
transmittance in, 215-216 
Absorbed photosynthetically active 
radiation (APAR), 325 
Absorptance 
hemispherical absorptance, 202 
radiometric concepts, 201 
terrain energy-matter 
interactions, 202-203 
Absorption band, 198 
Absorption of radiant energy, 198— 
200 
Absorption spectrum, of plant 
pigments, 317-318 
Active sensors, 10 


Adaptive Resonance Theory (ART), 
451, 453 
Additive color theory, 160 
ADS80 (Airborne Digital 
Sensor), 102 
Advanced Land Imager (ALI), for 
multispectral imaging, 73-75 
Advanced Spaceborne Thermal 
Emission and Reflection 
Radiometer (ASTER) 
multispectral imaging using linear 
arrays, 88-89, 598 
observing climate change, 23 
Advanced Very High Resolution 
Radiometer (AVHRR), 67-69, 
598 
Aerosol Free Vegetation Index 
(AFRJ), 331, 335-336 
AGL (Altitude Above Ground 
Level), 4 
AI. see Artificial Intelligence (AJ), in 
information extraction 
Airborne (suborbital) systems 
remote sensing measurement and, 4 
types of remote sensing 
systems, 20 
Airborne digital cameras 
digital oblique aerial 
photography, 105-106 
large-format cameras, 102-105 
medium-format cameras, 102 
overview of, 101 
small-format cameras, 101-102 
U.S. space shuttle photography, 
106-107 
Airborne Digital Sensor 
(ADS80), 102 


Airborne optical hyperspectral 


sensors, 460-461 


Airborne thermal-infrared 


hyperspectral sensors, 460-461 


Airborne Visible and Infrared 


Imaging Spectrometer 
(AVIRIS), 12, 14, 598 

imaging spectrometry using linear 
and area arrays, 96-97 

initial assessment of quality of 
hyperspectral data, 467 

pixel-by-pixel data correction, 
470 

selecting study area from flight 
lines, 465-466 

suborbital (airborne) systems, 20 


Algebra, for binary change detection 


asymmetric threshold, 522 

band ratioing, 519-520, 522 

image differencing, 518-519, 521 

Moving Threshold Windows, 
522-523 

overview of, 518—523 

statistical or symmetric 
thresholds, 522 


Algorithms 


approaches to problem 
solving, 431 
binary change detection, 514-518 
for change detection, 512-514 
image classification and, 399-402 
machine learning, 440-441, 444 
parametric and nonparametric, 
361-362 
spectral angle mapping, 479-480 


Algorithms, for supervised 


classification 
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Algorithms, for supervised 
classification (continued) 
maximum likelihood classifica- 
tion algorithm, 398-400 
maximum likelihood 
classification with prior 
probability information, 402 
maximum likelihood 
classification without prior 
probability information, 
400-401 
minimum distance to means clas- 
sification algorithm, 395-396 
nearest-neighbor classifiers, 
396-398 
overview of, 393 
parallelepiped classification 
algorithm, 393-395 
Algorithms, for unsupervised 
classification 
chain method for unsupervised 
classification, 402-406 
ISODATA method for 
unsupervised classification, 
406-411 
ALI (Advanced Land Imager), for 
multispectral imaging, 73-75 
Altitude Above Ground Level 
(AGL), 4 
American National Standards 
Institute (ANSI), spatial data 
standards, 130 
American Planning Association, 
Land-Based Classification 
Standard of, 365—366 
American Society for 
Photogrammetry and Remote 
Sensing (ASPRS), 3 
Analog (visual/” on-screen) 
visualization 
change detection, 514-518 
data analysis, 24—25 
Analog cameras, U.S. space shuttle 
photography, 106-107 
Analog image digitization 
digital image terminology and, 
37-38 
linear and area array charge- 
coupled device digitization, 
40-42 
microdensitometer digitization, 
38-39 
NAPP digitized data, 42-44 
overview of, 37, 599 
video digitization, 39 
Ancillary data 
data source for Date | in change 
detection, 543—544 


improving classification maps, 
422-423 
incorporating in classification 
process, 421-422 
Angular information, in remote 
sensing process, 18-20 
ANSI (American National 
Standards Institute), spatial 
data standards, 130 
AOIs (Areas of interest), 180-181 
APAR (absorbed photosynthetically 
active radiation), 325 
Archiving, data storage and, 123 
Area arrays 
charge-coupled device 
digitization, 40-42 
in imaging spectrometry. see 
Imaging spectrometry, using 
linear and area arrays 
Area measurement, display (output), 
180-181 
Areal sampling units, 558 
Areas of interest (AOIs), 180-181 
ART (Adaptive Resonance Theory), 
451, 453 
Artificial Intelligence (AJ), in 
information extraction 
advantages of artificial neural 
networks, 451-453 
classification based on 
human-derived rules, 435-436 
classification based on machine 
learning, 436-438 
from decision trees to production 
rules, 441-444 
decision-tree training and 
generation, 441 
defined, 429 
expert system user interface, 430 
expert systems applied to remote 
sensor data, 435 
feed forward multi-layer 
perceptron neural network 
with back propagation, 
448-450 
fuzzy ARTMAP neural 
network, 451 
inference engine, 434-435 
knowledge base creation, 430-434 
limitations of artificial neural 
networks, 453 
machine learning, 438-441 
mathematics of artificial neural 
networks, 447-448 
neural networks, 445-446 
online databases, 435 
overview of, 429-430 
random forest classifier, 444 


references, 453-457 
self-organizing map neural 
network, 450-451 
support vector machines, 444-445 
testing artificial neural 
networks, 447 
training artificial neural 
networks, 446—447 


Artificial neural networks (ANNs). 


see Neural networks 


ARVI (Atmospherically Resistant 


Vegetation Index), 331, 335 


Aspect effects, atmospheric 


correction, 230 


ASPRS (American Society for 


Photogrammetry and Remote 
Sensing), 3 


ASTER. see Advanced Spaceborne 


Thermal Emission and 
Reflection Radiometer 
(ASTER) 


Astrium, Inc. Sentinel-2 system, 


multispectral imaging using 
linear arrays, 87-88, 602 


Asymmetric threshold, in binary 


change detection, 522 


Asymmetry, of distribution, 141 
Atmosphere Removal (ATREM), 


469-471 


Atmospheric conditions, change 


detection considerations, 506 


Atmospheric correction 


absolute correction using ELC, 
220-223 

absolute radiometric correction 
of attenuation, 212-214 

based on radiative transfer model, 
216-220 

c correction, 231 

change detection and, 548-551 

correcting for slope and aspect 
effects, 230 

diffuse sky irradiance in absolute 
correction, 216 

local correlation filters, 232 

minnaert correction, 231 

multiple-date image 
normalization for relative 
correction, 224—230 

necessary corrections, 211-212 

overview of, 208 

relative radiometric correction of 
attenuation, 223-224 

single-image normalization for 
relative correction, 224 

statistical-empirical correction, 231 

target and path radiance in 
absolute correction, 214-215 


transmittance in absolute 
correction, 215-216 
unnecessary corrections, 208-211 
Atmospheric energy-matter 
interactions 
absorption of radiant energy, 
198-200 
overview of, 196, 204 
reflectance of radiation, 200-201 
refraction of light, 196-197 
scattering of radiation, 197-198 
Atmospheric transmission 
coefficients, 200 
Atmospheric windows, of radiant 
energy, 198 
Atmospherically Resistant 
Vegetation Index (ARVI), 
331, 335 
Atoms 
excitation of, 192 
ions and, 195 
ATREM (Atmosphere Removal), 
469-471 
Attenuation 
absolute correction of 
atmospheric, 212-214 
relative correction of 
atmospheric, 223-224 
AVHRR (Advanced Very High 
Resolution Radiometer), 
67-69 
AVIRIS. see Airborne Visible and 
Infrared Imaging Spectrometer 
(AVIRIS) 
Azimuthal (planar) map projections 
Azimuthal Equidistant map 
projections, 253-255, 260-261 
Lambert Azimuthal Equal-area 
maps, 255, 260-261 
overview of, 260 
perspective azimuthal maps, 260 
types of map projections, 
254-255, 260-261 


B 
Back-propagation (BP), feed 
forward multi-layer perceptron 
neural network with, 448-450, 
453 
Bad pixels, 205-206 
Band 
of electromagnetic spectrum, 190 
statistical examination of 
hyperspectral data, 467 
visual band examination of 
hyperspectral data, 465 
Band Interleaved by Line (BIL), 
digital image format, 108 


Band Interleaved by Pixel (BIP), 
digital image format, 108 
Band ratioing 
in binary change detection, 
519-520, 522 
image enhancement, 288-291 
Band Sequential (BSQ), digital 
image formats, 108 
Band-by-band spectral polishing, 
470-471 
Batch processing, of digital 
images, 120 
Bhattacharyya distance, in feature 
selection for image 
classification, 388 
Bidirectional reflectance distribution 
function (BRDF) 
angular information and, 19 
investigating with Multiangle 
Imaging Spectrometer, 20 
temporal resolution and, 16 
BIL (Band Interleaved by Line), 
digital image format, 108 
Bilinear interpolation, image-to-map 
rectification, 251 
Binary change detection 
analog “on-screen” visualization, 
514-518 
assessing map accuracy, 577-578 
image algebra for, 518 
overview of, 514 
Binary change mask, applied to 
Date 2 in change detection, 
544-545 
Binomial probability theory, 563 
Biogeochemical cycles, climate 
change and, 21 
Biophysical variables, remote 
sensing data collection and, 
10-11 
BIP (Band Interleaved by Pixel), 
digital image format, 108 
Bitmapped graphics, 157-160 
Black box, neural network as, 453 
Black-and-white brightness maps, 157 
Black-and-white hard-copy image 
display 
laser or ink-jet printer brightness 
maps, 156 
line printer and plotter brightness 
maps, 154-156 
overview of, 154 
Blackbodies 
electromagnetic radiation and, 
188 
radiation curves, 192 
Bohr, Niels, 193 
Boolean AND logic, 434 
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BP (back-propagation), feed 
forward multi-layer perceptron 
neural network with, 448-450, 
453 

BRDE. see Bidirectional reflectance 
distribution function (BRDF) 

Brightness maps 

black-and-white and color, 157 

defined, 153 

laser printers and ink-jet 

printers, 156 

line printers and plotters, 154-156 
Brightness values 

histograms of, 226 

PCA and, 310 

Brovey transform, RGB to IHS 
transformation, 173 

BSQ (Band Sequential), digital 
image formats, 108 


Cc 
C correction 
atmospheric correction, 231 
radiometric correction, 231 
C4.5/C5.0 learning algorithms 
decision tree generation, 441 
machine learning and, 440 
CAI (Cellulose Absorption Index), 
332, 339 
Cameras 
analog, 106-107 
digital. see Airborne digital 
cameras 
Career opportunities, 29-30 
CART (classification and regression 
tree) analysis, 423, 440 
Cartography 
image processing functions, 126 
mapping sciences, 5—6 
CartoSat, ISRO remote sensing 
systems, 84-85, 601 
CASI (Compact Airborne 
Spectrographic Imager- 
1500), 99, 600 
Cathode-ray tube (CRT), 156-157 
C-CAP (Coastal Change Analysis 
Program), 370-371, 595 
CCDC (Continuos Change 
Detection and Classification), 
528 
CCDs (charge-coupled devices) 
large-format digital cameras, 102 
linear and area array digitization, 
40-42 
CDs, rapid access mass storage, 
123-124 
Cellulose Absorption Index (CAI), 
332, 339 
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Central processing units (CPUs) 
history and efficiency of 
measurement, 112-114 
Moore’s Law and, 114 
overview of, 112 
Central tendency, measure of, 138 
Ceptometer, in-situ data collection, 
2-3 
CERES (Clouds and the Earth’s 
Radiant Energy System), 23 
Chain method 
assigning pixels to clusters, 
404-406 
cluster building, 403-404 
for unsupervised classification, 
402-403 
Change detection 
analog “on-screen” visualization 
in, 514-518 
ancillary data source used as 
Date 1, 543-544 
atmospheric correction, 548-551 
binary algorithms for, 514 
binary change mask applied to 
Date 2, 544-545 
chi-square transformation, 545 
cross-correlation, 545-546 
data analysis and, 27 
digital image processing 
software, 129 
environmental/developmental 
considerations, 506—512 
image algebra for, 518-523 
LiDAR, 531-532, 598 
map accuracy assessment, 
575-578 
multiple-date composite image, 
523-529 
Neighborhood Correlation 
Image, 538-542 
overview of, 501 
photogrammetric, 529-531 
post-classification comparison, 
532-538 
references, 551—555 
remote sensing considerations, 
504-506 
selecting algorithm for, 
512-514 
selecting hard or soft logic 
approach, 502, 504 
selecting land use/land cover 
classification system, 502 
selecting per-pixel or object-based 
approach, 504 
specifying geographic region of 
interest, 501-502 


specifying thematic attributes or 
indicators of interest, 501 
specifying time period for, 502 
spectral change vector analysis, 
542-543 
steps in, 503 
visual on-screen and digitization, 
546-548 
ChangeMatters, Esri Inc., 
516-518 
Channel, electromagnetic 
spectrum, 190 
Charge-coupled devices (CCDs) 
large-format digital cameras, 102 
linear and area array digitization, 
40-42 
Chi-square transformation, applied 
to change detection, 545 
Chloroplasts, in photosynthesis, 316 
Chromaticity color coordinate 
system, 172 
Classification. see also Pattern 
recognition 
based on human-derived rules, 
435-436 
based on machine learning, 
436-438 
schemes, 365 
supervised. see Supervised 
classification 
unsupervised. see Unsupervised 
classification 
Classification and regression tree 
(CART) analysis, 423, 440 
Classification learning, 433 
Climate change, 20-23 
Climate Continuity missions, 23 
Cloud computing, rapid access mass 
storage, 123-124 
Clouds, environmental/ 
developmental considerations 
in change detection, 506-507 
Clouds and the Earth’s Radiant 
Energy System (CERES), 23 
Cluster busting, for unsupervised 
classification, 412 
Cluster sampling, 567 
Clustering. see Unsupervised 
classification 
Coastal Change Analysis Program 
(C-CAP), 370-371, 595 
Coefficient of determination, 143 
Color 
additive and subtractive color 
theories, 160 
brightness maps, 157 
density sliced, 163 


display resolution and, 125 
look-up tables, 160-164 
Color composites 
assessing quality of hyperspectral 
data, 465 
OIF and, 164-167 
overview of, 164 
Sheffield Index, 167 
Color coordinate systems 
chromaticity color coordinate 
system, 172 
RGB. see Red-Green-Blue (RGB) 
Color criterion, of image 
objects, 414 
Color images. see Brightness maps 
Color space transformation, for 
image fusion, 169-173 
Column drop-outs 
partial, 207 
remote sensing correction and, 
205-207 
Compact Airborne Spectrographic 
Imager- 1500 (CASI), 99, 600 
Compass gradients masks 
for linear edge enhancement, 299 
for spatial filtering, 301 
Compilers, for digital image 
processing, 120-121 
Component substitution, for image 
fusion, 169-173 
Composites, color. see Color 
composites 
Compression, image file formats 
and, 157 
Computers, 115 
Conditional variance detection, in 
texture transformation, 
342-343 
Conditions 
in classification based on 
human-derived rules, 436 
in machine learning, 440 
Conduction, of energy, 186-187 
Confidence-building assessment, 558 
Conformal map projections, 
253, 255, 263-264 
Conical map projections 
Albers Equal-area Conic map 
projection, 255, 261-262 
Lambert Conformal Conic map 
projection, 255, 262-263 
overview of, 255, 261 
Continuos Change Detection and 
Classification (CCDC), 528 
Continuum removal (CR), in 
mapping and matching 
hyperspectral data, 484 


Contrast enhancement 
linear, 282-286 
nonlinear, 286-288 
overview of, 282 

Contrast stretching. see Contrast 
enhancement 

Control points. see Ground control 
points (GPCs) 

Convection, of energy, 186-187 

Correlation Matrix 

collecting training data, 379-382 
in feature selection for image 
classification, 388, 392 

Cosine correction, radiometric 
correction, 230-231 

Cospectral parallelepiped plots, 
graphic methods of feature 
selection, 383, 385 

Costs, software, 129-130 

Cowardin System, 371-373, 595 

CPUs. see Central processing units 
(CPUs) 

CR (continuum removal), in 
mapping and matching 
hyperspectral data, 484 

Crisp data. see Hard (or crisp) data 

Cross-correlation, change detection 
and, 545-546 

CRT (cathode-ray tube), 156-157 

Cultural vegetation, 371 

Cylindrical map projections, 
253-259 


D 
DACs (digital-to-analog converters) 
8-bit color processing and, 
161-162 
overview of, 157 
Data analysis, in remote sensing 
process, 23-27 
Data blunders, range of dispersion 
and, 139 
Data collection 
airborne digital cameras, 101 
ALI system, 73-75 
analog image digitization, 37 
ASTER system, 88-89, 598 
AVHRR system, 67-69, 598 
AVIRIS imaging spectrometer, 
96-97, 598 
Digital Globe, Inc. systems, 92 
digital image formats and, 108 
digital image terminology and, 
37-38 
digital oblique aerial 
photography, 105-106 
digitization considerations, 42-44 


DMC International Imaging Ltd. 
systems, 93-94 

EarthWatch, Inc. systems, 91—92 

EO-1 system, 73 

EROS systems, 92, 600 

ERTS and Landsat system, 48-51 

GOES system, 64—66 

hyperspectral systems, 460-462 

HyVista, Inc imaging 
spectrometer, 100-101 

imaging spectrometry using linear 
and area arrays, 94-96 

ISRO system, 82—85 

Itres, Inc. imaging spectrometers, 
99-100, 600 

KARI system, 85-87, 601 

Landsat MSS, 51-53, 599 

Landsat TM, 53-57, 599 

Landsat TM enhanced, 57—63, 599 

Landsat 8, 75-76, 599 

large-format digital cameras in, 
102-105 

LDCM system, 75-76 

linear and area array charge- 
coupled device digitization, 
40-42 

medium-format digital cameras 
in, 102 

microdensitometer digitization, 
38-39 

MISR system, 89-90 

MODIS imaging spectrometer, 
97-99, 599 

multispectral imaging using 
discrete detectors and scanning 
mirrors, 48 

multispectral imaging using linear 
arrays, 73 

multispectral remote sensing in, 
44-48 

multispectral systems, 460 

NAPP digitized data, 42-44, 586 

NASA EO-1 Hyperion 
hyperspectral imager, 96 

NASA HyspIRI imaging 
spectrometer, 98-99 

NOAA multispectral scanner 
sensors, 63 

overview of, 37 

panchromatic systems, 459-460 

Pleiades optical system, 81-82, 602 

RapidEye, Inc. systems, 93, 602 

remote sensing, 10-12 

SeaWiFS system, 70-73 

Sentinel-2 system, 87-88, 602 

small-format digital cameras in, 
101-102 
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Space Imaging, Inc systems, 90— 
91 
SPOT systems, 76-81, 602 
summary and references, 108-110 
Suomi NPP system, 69—70 
U.S. space shuttle photography, 
106-107 
video digitization, 39 
Data storage, for digital image 
processing, 122-123 
Databases, for expert system, 435 
Decadal Survey missions, observing 
climate change, 23 
Decision trees 
advantages of classifiers, 442-444 
classification based on human- 
derived rules, 435-436 
classification based on machine 
learning, 436-438 
classifiers in expert systems, 430 
in conceptualization of expert 
systems, 433-434 
from decision trees to production 
rules, 441-442 
machine learning, 438-441 
machine learning analysis of 
hyperspectral data, 487-491 
training and generation, 441 
Density sliced, color, 163 
Department of the Interior, Fish & 
Wildlife Service Classification 
of Wetlands and Deepwater 
Habitats of the United States, 
371-373 
Derivative spectroscopy, 494495 
Derived thematic maps. see 
Thematic map, accuracy of 
derived 
Design-based inference, 558 
Developable surfaces, creating map 
projections, 253-255 
Diffuse reflection, 200-201 
Diffuse sky irradiance, in absolute 
atmospheric correction, 216 
Digital cameras, airborne. see 
Airborne digital cameras 
Digital elevation model, in GIS, 7-8 
Digital Globe, Inc. systems, 92, 602 
Digital image formats, 108 
Digital image processing 
batch processing, 120 
career opportunities, 30 
computer types, 115 
CPUs, 112-114 
data analysis, 24-25 
data storage, 122—123 
display resolution, 123-125 
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Digital image processing (continued) 
GUIs, 119-120 
hardware considerations, 111-112 
image processing functions, 126 
input devices, 121 
mode of operation, 116-119 
NSDI and, 130 
operating systems and compilers, 
120-121 
output devices, 121-122 
overview of, 111 
references, 130 
ROM and RAM and, 116 
serial and parallel processing, 116 
software considerations, 125-126 
software costs, 129-130 
software programs, 126-129 
Digital image terminology, 37-38 
Digital Mapping Camera 
(DMC), 103 
Digital oblique aerial photography, 
105-106 
Digital remote sensor data collec- 
tion. see also Remote sensing 
multispectral remote sensing, 
44-48 
overview of, 44 
Digital surface models (DSMs), 
change detection and, 531 
DigitalGlobe, Inc, multispectral 
imaging using linear arrays, 
91-92, 602 
Digital-to-analog converters 
(DACs) 
8-bit color processing and, 
161-162 
overview of, 157 
Digitization 
analog. see Analog image 
digitization 
visual on-screen change detection 
and digitization, 546-548 
Dimensionality 
feature selection for image 
classification and, 382 
reducing dimensionality of 
hyperspectral data, 472-474 
Discrete detectors, in multispectral 
imaging. see Multispectral 
imaging, using discrete 
detectors and scanning mirrors 
Discrete multivariate techniques, 
applied to error matrices, 
570-571 
Dispersion, measures of, 138-141 
Display (output) 
area measurement, 180-181 
bitmapped graphics, 157-160 


black-and-white and color 
brightness maps, 157 
black-and-white hard-copy image 
display, 154 
color composites, 164-167 
color look-up tables, 160-164 
color space transformation and 
component substitution, 
169-173 
image display considerations, 
153-154 
laser or ink-jet printer brightness 
maps, 156 
length (distance) measurement, 
176-179 
line printer and plotter brightness 
maps, 154-156 
merging (fusing) remotely sensed 
data, 167-169 
overview of, 153 
PCA, ICA, or Gram-Schmidt 
substitution, 173-175 
perimeter measurement, 179-180 
pixel-by-pixel addition of high- 
frequency information, 175 
references, 181—183 
regression kriging for image 
fusion, 175 
resolution, 123-125 
RGB color coordinate system, 160 
SFIM for image fusion, 175 
shape measurement, 181 
simple band substitution for 
image fusion, 169 
temporary video displays, 156 
Display function, image 
processing, 126 
Distance measurement. see Length 
(distance) measurement 
Distributions 
measures of, 141 
sample size based on binomial or 
multinomial, 563—564 
in sampling, 132-133 
univariate statistics and, 138 
Divergence, as measure of statistical 
separability, 387-392 
DMC (Digital Mapping Camera), 
103 
DMC International Imaging Ltd. 
systems, multispectral imaging 
using linear arrays, 93-94 
DMC-NigeriaSat-2, multispectral 
imaging using linear arrays, 94 
DSMs (digital surface models), 
change detection and, 531 
DVDs, rapid access mass storage, 
123-124 


E 
Earth observation economics, 28-29 
Earth Observation-1 (EO-1) 
Hyperion hyperspectral imager, 96 
multispectral imaging using linear 
arrays, 73 
satellite remote sensing 
systems, 20 
Earth Remote Observation Satellites 
(EROS), multispectral imaging 
using linear arrays, 92, 95 
Earth resource analysis perspective, 
30-32 
Earth Resource Technology Satellite 
Program (ERTS), 48-51 
EarthWatch, Inc. systems 
GIS systems and, 90 
multispectral imaging using linear 
arrays, 91-92 
Edge enhancement 
linear, 298-299 
linear and nonlinear, 296 
nonlinear, 299-302 
in spatial domain, 298 
Edge-preserving median filters, 297 
Edge-preserving Spectral- 
Smoothing (EPSS), 341-342 
Eigenvalues, 311-312, 472 
Einstein, Albert, 191 
ELC. see Empirical line calibration 
(ELC) 
Electromagnetic radiation (EMR) 
overview of, 187 
particle model, 191-196 
principles in radiometric 
correction. see Radiometric 
correction 
remote sensing instruments 
recording, 7 
wave model, 187-191 
Electrons 
ions, 195 
orbits, 192-193 
Ellipse plots, graphic methods of 
feature selection, 383, 385 
Empirical line calibration (ELC) 
absolute atmospheric correction 
using, 220-223 
surface reflectance and, 225 
Empirical semivariogram 
characteristics of, 148 
example models, 149-150 
terminology, 151 
understanding spatial structure of 
regionalized variables, 147 
Endmember determination 
locating spectrally purest 
pixels, 474 


n-dimensional endmember 
visualization, 475-479 
pixel purity index mapping, 475 
Energy-matter interactions 
atmospheric. see Atmospheric 
energy-matter interactions 
at sensor systems, 204-205 
terrain. see Terrain energy-matter 
interactions 
Enhanced Vegetation Index (EVI) 
indices for hyperspectral data 
analysis, 492 
vegetation indices, 331, 336 
Ensemble learning algorithms, 444 
Environmental/developmental 
considerations, in change 
detection, 506-512 
EO-1. see Earth Observation-1 (EO-1) 
EOS satellite, observing climate 
change, 23 
Epidermis cells, in photosynthesis, 
316 
EPSS (edge-preserving Spectral- 
Smoothing), 341-342 
Equal-area (equivalent) map 
projections, 253, 255, 261-262 
Equidistant map projections, 
253-255, 261 
EROS (Earth Remote Observation 
Satellites), multispectral imag- 
ing using linear arrays, 92, 95 
Error 
atmospheric correction. see 
Atmospheric correction 
in data collection, 3 
external geometric. see External 
geometric errors 
geometric correction. 
see Geometric correction 
internal geometric. see Internal 
geometric errors 
radiometric correction. see 
Radiometric correction 
Sampling error, 132 
sources of, 558—561 
Error matrices 
descriptive evaluation of, 569-570 
discrete multivariate techniques 
applied to, 570-571 
fuzzification of, 571-575 
overview of, 561—562 
ERTS (Earth Resource Technology 
Satellite Program), 48—51 
Esri, Inc. ChangeMatters, 516-518 
Euclidean distance, 176-177, 
395-396 
EVI. see Enhanced Vegetation Index 
(EVI) 


Excitation, of atoms, 192 
Exhaustive quality, of classes in 
classification schemes, 365 
Exitance 
radiant exitance, 201 
radiant flux density and, 203 
Expert systems 
applying to remote sensor 
data, 435 
decision trees in 
conceptualization of, 433-434 
heuristic knowledge-based expert 
system approaches to problem 
solving, 431 
inference engine, 434435 
knowledge base creation, 430-431 
knowledge representation 
process, 432-433 
limitations of experts and 
expertise, 431-432 
online databases for, 435 
user interface, 430 
External errors, radiometric 
correction and, 185 
External geometric errors 
altitude change, 240-241 
attitude change, 240-242 
GCPs and, 242 
overview of, 240 
Extinction coefficient, 199 


F 
Fast Fourier transform (FFT), 
spatial filtering in frequency 
domain, 306 
Feathering, mosaicked images, 267 
Feature selection, for image 
classification 
Bhattacharyya distance, 388 
Correlation Matrix, 388, 392 
divergence as measure of 
statistical separability, 
387-392 
graphic methods, 382-386 
Jeffreys-Matusita distance, 388 
overview of, 382 
PCA feature selection, 392—393 
statistical methods, 386—387 
Feature space plots 
graphic methods of feature 
selection, 382—383 
graphical examination of 
statistical relationships, 145 
Features, defined, 145 
Federal Geographic Data 
Committee (FGDC) 
metadata standards, 134, 137 
spatial data standards, 130 
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Federal Geographic Data 
Committee, National 
Vegetation Classification 
Standard, 371 

Feed forward multi-layer perceptron 
neural network with back 
propagation, 448-450 

FFT (Fast Fourier transform), 
spatial filtering in frequency 
domain, 306 

FGDC. see Federal Geographic 
Data Committee (FGDC) 

File formats, 157 

First-order statistics, spatial domain, 
340-341 

Fish & Wildlife Service 
Classification of Wetlands and 
Deepwater Habitats of the 
United States. see Cowardin 
System 371-373, 595 

Foundation missions, observing 
climate change, 23 

Fourier transform 

separating images into spatial 
frequency components, 
302-306 

spatial filtering in frequency 
domain, 306-308 

Frequency 

electromagnetic radiation, 
187-189 

radio-wave spectrum, 191 

visible portion of electromagnetic 
spectrum, 190 

Frequency domain, spatial filtering 
in, 306-308 

Full Width at Half Maximum 
(FWHM), in multispectral 
remote sensing, 12 

Fuzzy ARTMAP neural network, 
451, 453 

Fuzzy classification 

compared with hard 
classification, 364 

data classification methods, 25, 
361-362 

observations about classification 
schemes, 376 

pattern recognition and, 
412-413 

Fuzzy data 

specifying data option for change 
detection, 502, 504 

from spectral Mixture Analysis 
(SMA), 413 

FWHM (Full Width at Half 
Maximum), in multispectral 
remote sensing, 12 
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G 
GEOBIA. see Geographic object- 
based image analysis (GEOBIA) 
Geodesy, mapping sciences, 5—6 
GeoEye, Inc. systems, multispectral 
imaging using linear arrays, 
91, 601 
Geographic Information Systems 
(GIS) 
commercial consortiums using 
information from, 90 
data analysis and, 26 
digital elevation model in, 7 
image processing functions, 126 
integrating with digital image 
processing, 129 
mapping sciences and, 5—6 
Geographic object-based image 
analysis (GEOBIA) 
defined, 362 
digital image processing 
software, 127 
image object metrics, 418-419 
image segmentation examples, 
415-417, 419-420 
overview of, 414-415 
Geometric correction 
azimuthal map projections, 
254-255, 260-261 
conformal map projections, 253, 
255, 263-264 
conical map projections, 255-256, 
261-263 
cylindrical map projections, 
253-259 
data analysis and, 24 
equal-area (equivalent) map 
projections, 253, 255, 261-262 
equidistant map projections, 
253-255, 261 
external geometric errors, 240-242 
GCPs, 262-267 
hybrid approach to image 
rectification/ registration, 
243-244 
of hyperspectral remote sensor 
data, 471-472 
image processing functions, 126 
Image-to-image registration, 243 
image-to-map rectification, 
242-243 
image-to-map rectification logic, 
244-252 
intensity interpolation resampling 
methods, 267 
internal geometric errors, 235-240 
Mercator map projections, 254-256 


mosaicking, 267-271 
references, 271—272 
selecting map projection for 
image-to-map rectification 
example, 253 
SPCS, 262 
types of, 242 
Geostationary Operational 
Environmental Satellite 
(GOES) 
GEOS Imager, 64—65 
GEOS Sounder, 65—67 
multispectral imaging using 
discrete detectors and scanning 
mirrors, 64 
Geostatistical analysis, 145-147, 
578-579 
GIF (graphic interchange format), 157 
Global Monitoring for Environment 
and Security (GMES), 87 
Global positioning system (GPS), in 
situ data collection and, 
2-3, 10 
GMES (Global Monitoring for 
Environment and Security), 87 
GOES. see Geostationary 
Operational Environmental 
Satellite (GOES) 
Goniometer, documenting changes 
in at-sensor radiance, 20 
GPCs. see Ground control points 
(GPCs) 
GPS (global positioning system), in 
situ data collection and, 2-3, 10 
Gram-Schmidt, substitution methods 
for image fusion, 173-175 
Graphic interchange format (GIF), 
157 
Graphic methods, in feature 
selection for image 
classification, 382—386 
Graphical user interfaces (GUIs), 
119-120 
Grayscale. see Brightness maps 
Ground control points (GPCs) 
change detection and, 531 
determining optimum geometric 
rectification, 262-264 
in geostatistical analysis, 145-147 
multiple regression coefficients in 
computing GPC total error, 
264-267 
overview of, 262 
Ground reference information 
error matrices and, 562 
response design for obtaining, 
568-569 


in situ data collection, 3 
training vs., 562-563 
GUIs (graphical user interfaces), 
119-120 


H 
Hard (or crisp) data 
classification methods, 361—362 
compared with fuzzy 
classification, 364 
data classification, 25 
specifying data option for change 
detection, 502, 504 
Hard-copy displays 
laser or ink-jet printer brightness 
maps, 156 
line printer and plotter brightness 
maps, 154-156 
printers for, 154 
Hardware 
batch processing, 120 
computer types, 115 
consideration related to image 
processing, 111-112 
CPUs, 112-114 
data storage, 122—123 
display resolution, 123-125 
GUIs, 119-120 
input devices, 121 
mode of operation, 116-119 
operating systems and compilers, 
120-121 
output devices, 121-122 
ROM and RAM and, 116 
serial and parallel processing, 116 
Hemispherical absorptance, terrain 
energy-matter interactions, 202 
Hemispherical reflectance, terrain 
energy-matter interactions, 
202-203 
Hemispherical transmittance, terrain 
energy-matter interactions, 202 
Heuristic knowledge, knowledge- 
based expert system 
approaches to problem 
solving, 431 
HFF (high-frequency filters), for 
spatial filtering, 297-298, 301 
Hierarchical decision tree 
classifiers, 433 
Hierarchical quality, of classes in 
classification schemes, 365 
High-frequency filters (HFF), for 
spatial filtering, 297-298, 301 
Histograms 
of brightness values, 136, 226 
of distributions, 133 


equalization in nonlinear contrast 
enhancement, 286 
in image quality assessment, 
133-134 
single-image normalization for 
relative atmospheric 
correction, 224 
Hue, intensity-hue-saturation (IHS), 
169-172 
Human-derived rules, classification 
based on, 435-436 
Humidity, change detection 
environmental/developmental 
considerations, 507 
Hybrid approach to image rectification/ 
registration, 243-244 
Hybrid variables, remote sensing 
data collection and, 10-11 
Hydrologic cycle, climate change 
and, 21 
HyMap, Whiskbroom hyperspectral 
scanner, 100-101, 601 
Hyperspectral remote sensing 
airborne optical sensors, 460-461 
airborne thermal-infrared 
sensors, 460-461 
data analysis and, 26 
data collection and, 460 
defined, 44 
digital image processing 
software, 127 
extracting information from 
hyperspectral data, 462-465 
Hyperion hyperspectral imager, 96 
ICA-based fusion for color 
display of hyperspectral 
images, 167 
indices for data analysis, 
491-494 
initial assessment of image 
quality, 465-467 
machine learning analysis of 
hyperspectral data, 487-491 
mapping and matching using 
hyperspectral data, 479 
overview of, 12 
reducing dimensionality of 
hyperspectral data, 472-474 
satellite sensors, 460 
selecting study area from flight 
lines, 465 
spectroscopic library matching 
techniques, 484-487 
support vector machines analysis 
of hyperspectral data, 491 
Whiskbroom hyperspectral 
scanner, 100-101 


Hypothesis 
in classification based on human- 
derived rules, 436 
in knowledge representation 
process, 433-434 
remote sensing process and, 8 
HyspIRI imaging spectrometer, 
98-99 
HyVista, Inc., 100-101, 601 


I 

ICA (Independent Component 
Analysis) 

color display of hyperspectral 
images, 167 

substitution methods for image 
fusion, 173-175 

IFFT (Inverse Fast Fourier 
transform), 306 

IFOV. see Instantaneous-Field-Of- 
View (IFOV) 

IFSAR (Interferometric Synthetic 
Aperture Radar), digital 
elevation models produced 
by, 8 

IGBP. see International Geosphere- 
Biosphere Program (IGBP) 

IHS (intensity-hue-saturation), 
transformation to/from RGB, 
169-173 

IKONOS systems 

multispectral imaging using linear 
arrays, 90-91, 601 

pan-sharpening, 169 

Image differencing 

in binary change detection, 521 

image algebra used for binary 
change detection, 518-519 

Image enhancement 

Aerosol Free Vegetation Index, 
335-336 

Atmospherically Resistant 
Vegetation Index, 335 

band ratioing, 288-291 

Cellulose Absorption Index, 339 

conditional variance detection, 
342-343 

contrast enhancement, 282 

data analysis and, 24 

edge enhancement in spatial 
domain, 298-302 

Edge-preserving Spectral- 
Smoothing variance texture, 
341-342 

Enhanced Vegetation Index, 336 

factors controlling leaf 
reflectance, 315 
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first-order statistics in spatial 
domain, 340-341 

Fourier transform, 302-306 

high-frequency spatial filtering, 
297-298 

image magnification, 274-276 

image reduction, 273-274 

interaction of light with pigments 
in palisade mesophyll cells, 
316-320 

Kauth-Thomas Tasseled Cap 
Transformation, 327, 329 

landscape ecology metrics, 
350-353 

Leaf Water Content Index, 
333-334 

linear contrast enhancement, 
282-286 

low-frequency spatial filtering, 
294-297 

MERIS Terrestrial Chlorophyll 
Index, 339-340 

middle infrared interaction with 
water in spongy mesophyll 
cells, 323-325 

min-max texture operator, 
343-344 

Moran’s / spatial autocorrelation 
metric, 344—345 

near infrared interaction within 
spongy mesophyll cells, 
320-323 

neighborhood raster operations, 
291-293 

nonlinear contrast enhancement, 
286-288 

Normalized Burn Ratio, 340 

Normalized Difference Built-up 
Index, 338 

Normalized Difference Moisture 
or Water Index, 332-333 

Normalized Difference Vegeta- 
tion Index, 325-327, 329 

overview of, 273 

PCA in, 308-314 

Perpendicular Vegetation 
Index, 333 

Photochemical Reflectance 
Index, 339 

Red-edge Position 
Determination, 339 

Reduced Simple Ratio, 336-337 

references, 351 

remote sensing using derived veg- 
etation indices, 325, 330-332 

second-order statistics in spatial 
domain, 345-348 
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n-dimensional endmember 
visualization, 475-479 


Image enhancement (continued) 
Simple Ratio use by vegetation 
indices, 325 azimuthal map projections, overview of, 459 
Soil Adjusted Vegetation Index, 254-255, 260-261 panchromatic systems, 
334-335 basic operations, 244 459-460 
Soil and Atmospherically computing root-mean-squared pixel purity index mapping, 474 
Resistant Vegetation error of inverse mapping radiometric calibration, 467-468 


Image-to-image registration, 243 
Image-to-map rectification 


Index, 335 function, 250-251 red-edge position based on 
spatial convolution filtering, conformal map projections, 253, derivative ratio, 496 

293-294 255, 263-264 reducing dimensionality of 
spatial filtering, 293 conical map projections, 255-256, hyperspectral data, 472-474 
spatial filtering in frequency 261-263 references, 496-500 


domain, 306-308 cubic convolution, 251 
spectral profiles, 279-282 cylindrical map projections, 
texture statistics based on 253-259 
semivariogram, 349-350 equal-area (equivalent) map 
texture transformation, 340 projections, 253, 255, 261-262 
texture units as elements of equidistant map projections, 
texture spectrum, 348-349 253-255, 261 
transects (spatial profiles), 275 GCPs, 262-267 
Transformed Absorption in input-to-output (forward) 
Reflectance Index, 337-338 mapping, 244-245 
Triangular Vegetation intensity interpolation resampling 
Index, 336 methods, 267 
Vegetation Adjusted Nighttime interpolation methods, 251 
Light Urban Index, 338-339 Mercator map projections, 254-256 
vegetation indices, 314-315 output-to-input (inverse) 


selecting study area from flight 
lines, 465 

in situ data collection, 468-469 

spectral angle mapping 
algorithms, 479-480 

spectroscopic library matching 
techniques, 484—487 

subpixel classification, 480-484 

Imaging spectrometry, using linear 

and area arrays 

AVIRIS imaging spectrometer, 
96-97, 598 

HyVista, Inc imaging 
spectrometer, 100-101, 601 

Itres, Inc. imaging spectrometers, 


vegetation suppression, 340 mapping, 245-250 99-100, 600 
Visible Atmospherically Resistant selecting map projection for MODIS imaging spectrometer, 
Index, 338 image-to-map rectification 97-99, 599 


Image file formats, 157 
Image magnification, 274-276 
Image metadata, 579 
Image objects SPCS, 262 
in GEOBIA, 414 types of geometric correction, 
metrics, 418-419 242-243 
Image processing software. see Imaging spectrometry 
Digital image processing absolute atmospheric correction, 
Image processor frame buffer, 157 469-471 
Image quality assessment and continuum removal, 484 
statistics derivative spectroscopy, 494-495 
evaluating individual pixel values, endmember determination, 474 


example, 253 
spatial interpolation using coordi- 
nate transformation, 244 


NASA EO-1 Hyperion 
hyperspectral imager, 96 

NASA HyspIRI imaging 
spectrometer, 98-99 

overview of, 48, 94—96 

In situ data collection. see also 

Remote sensing 

collecting training data, 378 

data requirements, 9-10 

defined, | 

ground reference data, 3 

overview of, 1-3 


137-138 
feature space plots, 145 
geostatistical analysis, 145-147 
histograms in, 133-134 
Kriging process, 146-151 


mathematical notations, 131-132 


metadata, 134-137 
multivariate statistics, 141-145 
overview of, 131 

references, 151—152 

sampling theory and, 132-133 
univariate statistics, 138-141 


extracting information from 
hyperspectral data, 462-465 

geometric correction of 
hyperspectral data, 471-472 

hyperspectral systems, 460-462 

indices for hyperspectral data 
analysis, 491-494 

initial assessment of image 
quality, 465-467 

machine learning analysis of 
hyperspectral data, 487-491 

mapping and matching using 


remote sensing process and, 8 
removing biases, 7 


Independent Component Analysis 


(ICA) 

color display of hyperspectral 
images, 167 

substitution methods for image 
fusion, 173-175 


Index of refraction, 196 
Indian Space Research Organization 


(ISRO) 
CartoSat, 84-85, 601 


Image reduction, 273-274 

ImageSat International, Inc., 
multispectral imaging using 
linear arrays, 92, 600 


hyperspectral data, 479 
multispectral systems, 460 
narrow-band derivative-based 

vegetation indices, 495-496 


IRS-1A, 1B, 1C, and 1D, 83-84, 601 

multispectral imaging using linear 
arrays, 82-83, 601 

ResourceSat, 85, 601 


Indicators of interest, specifying in 
change detection process, 501 
Inductive learning, machine learning 
and, 438 
Inference, model-based and design- 
based, 558 
Inference engine 
in classification based on human- 
derived rules, 436 
for expert system, 434-435 
Information classes 
spectral classes compared 
with, 365 
unsupervised classification 
and, 402 
Information extraction, image 
processing functions, 126 
Information presentation, 27—28 
Ink-jet printer brightness 
maps, 156 
Input devices, for digital image 
processing, 121 
Instantaneous-Field-Of-View (IFOV) 
remote sensing measurement 
and, 4 
satellite systems and, 14 
Intensity interpolation 
image-to-map rectification, 251 
resampling methods, 267 
Intensity-hue-saturation (IHS), 
transformation to/from RGB, 
169-173 
Interferometric Synthetic Aperture 
Radar (IFSAR), digital 
elevation models produced 
by, 8 
Internal errors, radiometric 
correction and, 185 
Internal geometric errors 
image offset (skew) due to earth 
rotation, 235-236 
overview of, 235 
scanning system one-dimensional 
relief displacement, 239 
scanning system tangential scale 
distortion, 240 
scanning system-induced 
variation in ground resolution 
cell size, 236-239 
International Geosphere-Biosphere 
Program (IGBP) 
climate change and, 20 
Land-Cover Classification 
System, 374-375 
International Space Station 
(ISS), 107 
Inverse Fast Fourier transform 
(IFFT), 306 


Inverse-Distance-Weighting, 
Kriging process and, 147 
Ions, 195 
Irradiance, radiant flux density 
and, 203 
ISODATA. see Iterative Self- 
Organizing Data Analysis 
Technique (ISODATA) 
ISRO. see Indian Space Research 
Organization (ISRO) 
ISS (International Space 
Station), 107 
Iterative Self-Organizing Data 
Analysis Technique 
(ISODATA) 
examples, 409-411 
first iteration, 408-409 
initial arbitrary cluster allocation, 
407-408 
overview of, 406-407 
second to Mth iteration, 409 
Itres, Inc., imaging spectrometry 
using linear and area arrays, 
99-100, 600 


J 

Jeffreys-Matusita distance, in 
feature selection for image 
classification, 388 

Joint photographic experts group 
(JPEG), 157 


K 

Kappa analysis, 570-571 

Karhunen-Loeve analysis. see 
Principal Component Analysis 
(PCA) 

KARI (Korean Aerospace Research 
Institute), multispectral 
imaging using linear arrays, 
85-87, 601 

Kauth-Thomas Tasseled Cap 
Transformation, vegetation 
index, 327, 329, 330 

Kirsch nonlinear edge 
enhancement, 300 

Knowledge acquisition bottlenecks, 
problems with expert 
systems, 438 

Knowledge base, creating for expert 
system 

algorithmic approaches to 
problem solving, 431 

decision trees in conceptualiza- 
tion of expert systems, 433-434 

heuristic knowledge-based expert 
system approaches to problem 
solving, 431 
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knowledge representation 
process, 432-433 
limitations of experts and 
expertise, 431-432 
overview of, 430-431 
Knowledge representation process, 
432-433 
Coonan’s self-organizing map 
(SOM), 450-451, 453 
Korea Multi-Purpose Satellite 
(KOMPSAT), multispectral 
imaging using linear arrays, 
85-87, 601 
Korean Aerospace Research 
Institute (KARI), 
multispectral imaging using 
linear arrays, 85-87, 601 
Krieg, Daniel, 145-147 
Kriging process 
overview of, 146-151 
regression kriging for 
image fusion, 175 
Kurtosis, in normal distribution, 141 


L 
Laboratory, for digital image 
processing, 118-119 
LAI (leaf area index), 2-3, 325, 388 
Lambertian surface, reflection of 
energy by, 201 
Land cover, 364. see also Land-use/ 
land-cover classification 
schemes 
Land Remote Sensing Policy Act 
(1992), 57 
Land use, 364. see also Land-use/ 
land-cover classification 
schemes 
Land-Based Classification Standard 
(LBCS), 365-366 
Land-Cover Classification System, 
374-375 
Landsat Data Continuity Mission 
(LDCM), 75-76 
Landsat systems 
Landsat MSS, 20, 51-53, 599 
Landsat TM, 7, 53-57, 599 
Landsat TM enhanced, 57—63, 599 
Landsat 8, 75—76, 599 
multispectral imaging using 
discrete detectors and scanning 
mirrors, 48-51 
Landscape ecology metrics, 350-353 
Land-use/land-cover classification 
schemes 
Coastal Change Analysis 
Program, 370-371, 595 
Cowardin System, 371-373 
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Land-use/land-cover classification 
schemes (continued) 
Land-Based Classification 
Standard, 365-366 
Land-Cover Classification 
System, 374-375 
Land-Use/Land-Cover Classifica- 
tion System for Use with 
Remote Sensor Data, 366-367 
National Land Cover Database 
classification system, 367-370 
National Vegetation 
Classification Standard, 371 
observations about classification 
schemes, 375-376 
overview of, 364-365 
specifying for change 
detection, 502 
for use with remote sensor data, 
366-367 
Laplacian filters 
for linear edge enhancement, 299 
for spatial filtering, 301 
Large-format digital cameras, 102-105 
Laser printer brightness maps, 156 
LBCS (Land-Based Classification 
Standard), 365-366 
LCD (liquid crystal display), 156-157 
LDCM (Landsat Data Continuity 
Mission), 75—76 
Leaf area index (LAI), 2—3, 325, 388 
Leaf reflectance 
additive, 321-323 
factors controlling, 315 
Leaf Water Content Index (LWCI), 
330, 333-334 
Least-squares linear regression, 146. 
see also Kriging process 
Leica Geosystems, Ag., 102 
Length (distance) measurement 
Euclidean distance, 176-177 
Manhattan distance, 177-179 
overview of, 176 
LFF (low-frequency) filters, for 
spatial filtering, 294-297, 301 
LiDAR. see Light Detection and 
Ranging (LiDAR) 
Light Detection and Ranging 
(LiDAR), 599 
change detection, 531-532 
digital elevation models produced 
by, 7-8 
digital image processing 
software, 127 
intrusive impact on data 
collection, 8 
remote sensing instruments, 4 


Line drop-outs 
partial, 207 
remote sensing correction, 205—207 
Line printer brightness maps, 154-156 
Lineage documentation, of remote 
sensing-derived products, 
579-580 
Linear arrays 
charge-coupled device 
digitization, 40-42 
in imaging spectrometry. 
see Imaging spectrometry, 
using linear and area arrays 
in multispectral imaging. see 
Multispectral imaging, using 
linear arrays 
Linear contrast enhancement 
minimum-maximum contrast 
stretch, 282-284 
overview of, 282 
percentage linear and standard 
deviation contrast stretching, 
284-286 
piecewise linear contrast 
stretching, 284-286 
Linear edge enhancement, 298-299 
Linear spatial filters, 293 
Linear spectral unmixing, 480-484 
Line-start problems, remote sensing 
correction, 207-208 
Liquid crystal display (LCD), 156-157 
Local correlation filters, 
atmospheric correction, 232 
Look angles, change detection 
considerations, 505 
Look-up tables (LUTs) 
24-bit color processing, 164 
color look-up tables, 160-164 
overview of, 157 
Lossless compression, 157 
Lossy compression, 157 
Low-frequency (low-pass) filters 
(LFF), for spatial filtering, 
294-297, 301 
LUTs. see Look-up tables 
(LUTs) 
LWCI (Leaf Water Content Index), 
330, 333-334 


M 
Machine learning 
analysis of hyperspectral data, 
487-491 
classification based on, 436-438 
overview of, 438-441 
Mainframe (super) computers, 115 
Manhattan distance, 177-179 


Map projections 
azimuthal, 254-256, 260-261 
conformal, 253, 255, 262-263 
conical, 253-256, 261-263 
cylindrical, 253-259 
equal-area (equivalent), 253, 255, 
260-262 
equidistant, 253-255, 261 
Mercator, 254-257 
selecting for image-to- 
map rectification 
example, 253 
State Plane, 263 
UTM, 257-259 
MASI-600 (mid-wavelength 
infrared-600), imaging 
spectrometry using linear and 
area arrays, 99-100, 600 
Mathematics 
of artificial neural networks, 
447-448 
notation for image processing, 
131-132 
Maximum filters, for spatial 
filtering, 297, 301 
Maximum likelihood classification 
algorithm 
in machine learning, 444 
overview of, 398—400 
with prior probability 
information, 402 
without prior probability 
information, 400-401 
MCARII (Modified Chlorophyll 
Absorption Ratio Index), 494 
MDA Information Systems, Inc., 
526-528 
Mean 
measure of central tendency, 138 
in sampling, 133 
univariate statistics, 142, 144 
Measurement-device calibration 
error, in data collection, 3 
Measurements of Pollution in the 
Troposphere (MOPITT), 
observing climate change, 23 
Median 
measure of central tendency, 138 
in sampling, 133 
Median filters, for spatial filtering, 
295, 301 
Medium Resolution Imaging 
Spectrometer (MERIS), 494 
Medium-format digital cameras, 102 
Mercator map projections 
Space Oblique Mercator (SOM) 
map projection, 255, 259 


types of map projections, 253-256 
Universal Transverse Mercator 
(UTM) map projection, 255, 
257-259 
Merging (fusing) remotely sensed 
data 
color space transformation and 
component substitution, 169— 
173 
overview of, 167-169 
PCA, ICA, or Gram-Schmidt 
substitution, 173-175 
pixel-by-pixel addition of high- 
frequency information, 175 
regression kriging for image 
fusion, 175 
SFIM for image fusion, 175 
simple band substitution for 
image fusion, 169 
MERIS (Medium Resolution 
Imaging Spectrometer), 
index for hyperspectral data 
analysis, 494 
MERIS Terrestrial Chlorophyll 
Index (MTCI), 332, 339-340 
Mesophyll cells 
interaction of light with pigments 
in palisade mesophyll cells, 
316-320 
middle infrared interaction with 
water in spongy mesophyll 
cells, 323-325 
near infrared interaction within 
spongy mesophyll cells, 320-323 
in photosynthesis, 316 
Metadata 
image processing functions, 126 
image quality assessment, 
134-137 
remote sensing process and, 8 
Method-produced error, in data 
collection, 3 
Microdensitometer digitization, 38-39 
Microsoft, Inc., 104, 589 
Mid-wavelength infrared-600 
(MASI-600), imaging 
spectrometry using linear and 
area arrays, 99-100, 600 
Mie scattering, atmospheric energy- 
matter interactions, 198 
Minimum distance to means 
classification algorithm, 
395-396 
Minimum filters, for spatial filtering, 
297, 301 
Minimum noise fraction (MNF) 
PCA and, 309 


reducing dimensionality of 
hyperspectral data, 472 
Minimum-maximum contrast 
stretch, 284 
Min-max texture operator, 343-344 
Minnaert correction, 231 
MISR (Multiangle Imaging 
Spectroradiometer), 20, 89-90 
MLP (Multi-layer perceptron), 
448-450, 453 
MNF (minimum noise fraction) 
PCA and, 309 
reducing dimensionality of 
hyperspectral data, 472 
Mode 
measure of central tendency, 138 
in sampling, 133 
Mode of operation, for digital image 
processing, 116-119 
Model-based inference, 558 
Moderate Resolution Imaging 
Spectrometer (MODIS) 
imaging spectrometry using linear 
and area arrays, 97-99, 599 
Land-Cover Classification 
System and, 374-375 
observing climate change, 23 
Modified Chlorophyll Absorption 
Ratio Index (MCARI]1), 494 
Moore, Gordon, 113-114 
MOPITT (Measurements of 
Pollution in the Troposphere), 
observing climate change, 23 
Moran’s / spatial autocorrelation 
metric, as texture measure, 
344-345 
Mosaicking 
overview of, 267 
rectified images, 267-271 
Moving Threshold Windows 
(MTW), in binary change 
detection, 
522-523 
MRLC (Multi-Resolution Land 
Characteristics), 367 
MTCI. see MERIS Terrestrial 
Chlorophyll Index (MTCI) 
Multiangle Imaging 
Spectroradiometer (MISR), 
20, 89-90 
Multi-layer perceptron (MLP), 
448-450, 453 
Multimodal distribution, 133 
Multinominal distribution, 563—564 
Multiple-date composite images, in 
change detection 
CCDC, 528 
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NUCTI, 526-528 
overview of, 523 
PCA, 523-526 
supervised and unsupervised 
classification, 523 
Multiple-date image normalization, 
for relative atmospheric 
correction, 224-230 
Multi-Resolution Land 
Characteristics (MRLC), 367 
Multispectral imaging, using discrete 
detectors and scanning mirrors 
AVHRR system, 67-69, 598 
ERTS and Landsat system, 48-51 
GOES system, 64-66 
Landsat MSS, 51-53, 599 
Landsat TM, 53-57, 599 
Landsat TM enhanced, 57—63, 599 
NOAA multispectral scanner 
sensors, 63 
overview of, 47-48 
SeaWiFS system, 70-73 
Suomi NPP system, 69-70 
Multispectral imaging, using linear 
arrays 
ALI system, 73-75 
ASTER system, 88-89 
Astrium, Inc. Sentinel-2 system, 
87-88, 602 
Digital Globe, Inc. systems, 92 
DMC International Imaging Ltd. 
systems, 93—94 
EarthWatch, Inc. systems, 91-92 
EROS systems, 92, 600 
GeoE ye, Inc. systems, 90-91, 601 
ImageSat International, Inc., 
92, 600 
ISRO system, 82—85 
KARI system, 85-87, 601 
LDCM system, 75-76 
MISR system, 89-90 
overview of, 48 
Pleiades optical system, 81-82, 602 
RapidEye, Inc. systems, 93, 602 
SPOT systems, 76-81, 602 
Multispectral remote sensing 
data collection and, 12 
defined, 44 
digital data collection, 44-48 
digital image processing software, 
127 
Multivariate statistics 
covariance, 142-144 
correlation, 143-145 
overview of, 141-142 
Mutually exclusive quality, of classes 
in classification schemes, 365 
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N 
Narrow-band derivative-based 
vegetation indices, 495-496 
National Aerial Photography 
Program (NAPP) 
digitized data, 42-44, 586 
metadata standards, 134 
National Aeronautics & Space 
Administration (NASA) 
ALI system, 73-75 
ASTER system, 88-89, 593, 598 
AVHRR system, 66-69, 598 
AVIRIS imaging spectrometer, 
96-97 
EO-1 Hyperion hyperspectral 
imager, 96 
EO-1 system, 73 
GEOS Imager, 64-65 
GEOS Sounder, 65-66 
GEOS system, 64 
HyspIRI imaging spectrometer, 
98-99 
Landsat 8, 75-76, 598 
LDCM system, 75 
MISR system, 89-90 
MODIS imaging spectrometer, 
97, 599 
multispectral scanner sensors, 
48, 598 
National Land Cover Database 
(NLCD) classification system, 
367-370, 595 
National Oceanic and Atmospheric 
Administration (NOAA) 
Coastal Change Analysis 
Program, 370-371 
multispectral imaging using 
discrete detectors and scanning 
mirrors, 63 
Suomi NPP system, 69, 598 
National Spatial Data Infrastructure 
(NSDI) 
metadata standards, 134 
spatial data standards, 130 
National Urban Change Indicator 
(NUCD, multiple-date 
composite images in change 
detection, 526—528 
National Vegetation Classification 
Standard (NVCS), 371 
Natural vegetation, defined, 371 
NBR (Normalized Burn Ratio), 
332, 340 
NDBI (Normalized Difference 
Built-up Index), 332, 338 
NDVI. see Normalized Difference 
Vegetation Index (NDVI) 


NDWI (Normalized Difference 
Water Index), 493 
NDWI/NDMI (Normalized 
Difference Moisture or Water 
Index), 330, 332-333 
Nearest neighbor 
classifiers, 396-398 
image-to-map rectification, 251 
Neighborhood Correlation Image 
(NCI), 538-542 
Neighborhood raster operations 
qualitative modeling, 292-293 
quantitative modeling, 293 
Neural networks 
advantages of, 451-453 
feed forward multi-layer 
perceptron neural network with 
back propagation, 448-450 
fuzzy ARTMAP neural 
network, 451 
limitations of, 453 
mathematics of, 447-448 
overview of, 445-446 
self-organizing map neural 
network, 450-451 
testing, 447 
training, 446-447 
Newton, Sir Isaac, 191 
NLCD (National Land Cover 
Database) classification 
system, 367-370, 595 
N-line striping, remote sensing 
correction, 207—208 
NOAA. see National Oceanic and 
Atmospheric Administration 
(NOAA) 
Nonlinear contrast enhancement, 
286-288 
Nonlinear edge enhancement, 
299-302 
Nonmetric information extraction, 
data analysis and, 26 
Nonparametric information 
extraction, data analysis and, 26 
Nonparametric statistics 
algorithms based on, 361—362 
selecting classification 
algorithm, 393 
Normal distribution, in 
sampling, 132 
Normalization 
multiple-date image normaliza- 
tion for relative atmospheric 
correction, 224-230 
single-image normalization for 
relative atmospheric 
correction, 224 


Z 


Normalized Burn Ratio (NBR), 

332, 340 

Normalized Difference Built-up 

Index (NDBIJ), 332, 338 

Normalized Difference Moisture or 

Water Index (NDWI/NDMI), 

330, 332-333 

Normalized Difference Vegetation 
Index (NDVI) 

indices for hyperspectral data 
analysis, 492-493 

overview of, 325-327 

vegetation index, 329-330 

Normalized Difference Water Index 

(NDWI), 493 

NPOESS Preparatory Project 

(NPP), 69-70, 600 

NPP (NPOESS Preparatory 

Project), 69-70, 601 

NSDI. see National Spatial Data 

Infrastructure (NSDI) 

NUCI (National Urban Change 
Indicator), multiple-date 
composite images in change 
detection, 526-528 

Nugget effect, empirical 
semivariogram, 148 

NVCS (National Vegetation 

Classification Standard), 371 


O 
Object-based change detection 
(OBCD), 504 
Object-based image analysis 
(OBIA) 
assessing accuracy of OBIA 
classification map, 578 
geographic object-based image 
analysis (GEOBIA) 
classification, 414-421 
overview of, 413-414 
post-classification comparison 
change detection, 534-538 
Obscuration, change detection 
environmental/developmental 
considerations, 510 
OIF (optimum index factor), color 
composites and, 
164-167 
OLI (Operational Land Imager), 
multispectral imaging using 
linear arrays, 75 
Olympic filter, 297 
Online databases, for expert 
system, 435 
Open-source software, 129 
Operating systems (OSs), 120-121 


Operational Land Imager (OLD, 
multispectral imaging using 
linear arrays, 75 

Optical hyperspectral sensors, 
airborne, 460-461 

Optimum index factor (OIF), color 
composites and, 164-167 

Orbimage, Inc., 90 

Orbital platforms, remote sensing 
measurement and, 4 

OSs (operating systems), 120-121 

Output devices, for digital image 
processing, 121-122 


P 
Pan-sharpening, 169 
Parallel processing, in digital image 
processing, 116 
Parallelepiped classification 
algorithm, 393-395 
Parallelepiped plots, graphic 
methods of feature selection, 
383, 385 
Parametric information extraction, 
24-25 
Parametric statistics 
algorithms based on, 361-362 
selecting classification 
algorithm, 393 
Partial column drop-outs, 207 
Partial line drop-outs, 207 
Particle model, of electromagnetic 
energy, 191-196 
Particular matter, 191 
Passive sensors, 10 
Path radiance, in absolute 
atmospheric correction, 
214-215 
Pattern recognition 
algorithm selection for image 
classification, 393 
Bhattacharyya distance, 388 
chain method for unsupervised 
classification, 402-406 
Coastal Change Analysis 
Program, 370-371 
Correlation Matrix feature 
selection, 388, 392 
Cowardin System, 371-373 
divergence as measure of 
statistical separability, 387-392 
feature selection for image 
classification, 382 
fuzzy classification, 412-413 
geographic object-based image 
analysis and classification, 
414-42] 


graphic methods of feature 
selection, 382—386 

improving classification maps by 
incorporating ancillary data, 
422-493 

incorporating ancillary data in 
classification process, 
421-422 

ISODATA method for 
unsupervised classification, 
406-411 

Jeffreys-Matusita distance, 388 

Land-Based Classification 
Standard, 365-366 

Land-Cover Classification 
System, 374-375 

land-use/land-cover classification 
schemes, 364—367 

maximum likelihood 
classification algorithm, 
398-400 

maximum likelihood 
classification with prior 
probability information, 402 

maximum likelihood 
classification without prior 
probability information, 
400-401 

minimum distance to means 
classification algorithm, 
395-396 

National Land Cover Database 
classification system, 367-370 

National Vegetation 
Classification Standard, 371 

nearest-neighbor classifiers, 
396-398 

object-based image analysis 
classification, 413-414 

observations about classification 
schemes, 375-376 

overview of, 361-362 

parallelepiped classification 
algorithm, 393-395 

PCA feature selection, 392—393 

references, 423-427 

selecting classification 
algorithm, 393 

statistical methods of feature 
selection, 386—387 

supervised classification, 362-364 

training site selection and 
statistics extraction, 376-382 

unsupervised classification, 402 

unsupervised cluster busting, 412 

PCA. see Principal Component 

Analysis (PCA) 
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PCs (Personal computers), 115 
Pearson’s product-moment 
correlation coefficient, 143 
Perimeter measurement, display 
(output), 179-180 
Perpendicular Vegetation Index 
(PVI), 330, 333 
Per-pixel 
data classification, 25, 362 
logical channel classification, 422 
object-oriented classification 
compared with, 420 
post-classification comparison 
change detection, 533-534 
selecting per-pixel or object-based 
approach to change 
detection, 504 
Personal computers (PCs), 115 
Petiole, in water transport in 
plants, 323 
Phenological cycles 
change detection environmental/ 
developmental considerations, 
508-509 
example of temporal resolution, 16 
urban-suburban phenological 
cycles, 509-510 
Photochemical Reflectance Index 
(PRI), 332, 339, 492-493 
Photoelectric effect, 195 
Photogrammetry 
change detection, 529-531 
data analysis and, 24 
mapping software, 129 
remote sensing instruments, 3 
Photographic interpretation, 3 
Photons, 191, 193 
Photosynthesis, 316 
Photosynthetically active radiation 
(PAR), 318 
Pictometry International, Inc., 
105-106, 601 
PIFs (Pseudo-invariant features), 
multiple-date image 
normalization for relative 
atmospheric correction, 
224, 227 
Pigments, absorption spectrum for, 
317-318 
Pixels 
endmember determination, 474 
evaluating brightness values, 
137-138 
neighborhood function in 
GEOBIA, 415 
per-pixel analysis. see Per-pixel 
pixel purity index mapping, 475 
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Pixels (continued) 
pixel-by-pixel addition of high- 
frequency information, 175 
random bad pixels (shot noise), 
205-206 
subpixel classification, 480-484 
Planar map projections. see 
Azimuthal (planar) map 
projections 
Planck, Max, 193 
Pleiades optical system, 81-82, 602 
Plotter brightness maps, 154-156 
Polarization information, in remote 
sensing process, 17-18 
Populations 
population-based error matrix, 561 
in sampling, 132 
Post-classification comparison 
change detection 
OBIA, 534-538 
overview of, 532 
per-pixel, 533-534 
PRI (Photochemical Reflectance 
Index), 332, 339, 492-493 
Principal Component Analysis 
(PCA) 
brightness values and, 310 
in feature selection for image 
classification, 392-393 
in image enhancement, 308-314 
multiple-date composite images 
in change detection, 523-526 
substitution methods for image 
fusion, 173-175 
Thematic Mapper statistics, 309 
Printers 
brightness maps, 154-156 
for hard-copy displays, 154 
Probability density functions, 
algorithm selection for image 
classification, 399-402 
Producer’s accuracy, 569 
Production rules 
from decision trees to, 441-444 
in machine learning, 440 
Programs. see Software 
Pseudo-invariant features (PIFs), 
multiple-date image normal- 
ization for relative atmo- 
spheric correction, 224, 227 
PVI (Perpendicular Vegetation 
Index), 330, 333 
Pythagorean theorem, 176 


Q 
Quantas, of energy, 191 
Quantum leap, 193 


Quantum theory (Planck and Bohr), 
193-195 

QuickBird, multispectral imaging 
using linear arrays, 92, 602 


R 
R Development Core Team, 
machine learning and, 440 
RADAR. see Radio Detection and 
Ranging (RADAR) 
Radiance 
radiant flux density and, 203-204 
radiometric concepts, 201 
Radiant energy, 196, 201 
Radiant exitance, radiometric 
concepts, 201 
Radiant flux density, terrain energy- 
matter interactions, 203-204 
Radiant flux, terrain energy-matter 
interactions, 201-202 
Radiation 
electromagnetic models, 187 
of energy, 186-187 
particle models, 191-196 
Radiation budget equation, 201 
Radiative transfer model, 216-220 
Radiative Transfer-based Absolute 
Atmospheric Correction, 
469-471 
Radio Detection and Ranging 
(RADAR) 
digital elevation models produced 
by, 8 
digital image processing 
software, 129 
intrusive impact on data 
collection, 8 
remote sensing instruments, 4 
Radiometric correction 
absolute atmospheric correction 
of attenuation, 212-214 
absolute atmospheric correction 
of diffuse sky irradiance, 216 
absolute atmospheric correction 
of spectral data, 469-471 
absolute atmospheric correction 
of transmittance, 215-216 
absolute atmospheric correction 
using ELC, 220-223 
absorption of radiant energy, 
198-200 
atmospheric correction, 208, 212 
atmospheric correction based on 
radiative transfer model, 
216-220 
atmospheric energy-matter 
interactions, 196, 204 


c correction, 231 

conduction, convection, and 
radiation, 186—187 

cosine correction, 230-231 

data analysis and, 24 

electromagnetic energy 
interactions, 186 

electromagnetic radiation 
models, 187 

energy-matter interactions at 
sensor systems, 204-205 

hemispherical reflectance, 
absorptance, and 
transmittance, 202—203 

image processing functions, 126 

line or column drop-outs, 
205-207 

line-start problems, 207 

local correlation filters, 232 

minnaert correction, 231 

necessary atmospheric 
corrections, 211—212 

n-line striping, 207-208 

overview of, 185—186, 467-468 

particle model of electromagnetic 
energy, 191-196 

radiant flux density, 203-204 

random bad pixels 
(shot noise), 205 

references, 232—234 

reflectance of radiation, 200-201 

refraction of light, 196-197 

relative atmospheric correction of 
attenuation, 223-224 

relative atmospheric correction 
using multiple-date image 
normalization, 224-230 

relative atmospheric correction 
using single-image 
normalization, 224 

scattering of radiation, 197-198 

in situ collection of spectral data, 
468-469 

slope and aspect effects, 230 

statistical-empirical 
correction, 231 

target and path radiance in 
absolute atmospheric 
correction, 214-215 

terrain energy-matter 
interactions, 201—202 

unnecessary atmospheric 
corrections, 208—211 

wave model of electromagnetic 
energy, 187-191 

Radiometric rectification technique, 
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Radiometric resolution 
change detection considerations, 
506 
in remote sensing process, 17 
RAID (redundant arrays of inexpen- 
sive hard disks), 122-123 
RAM (Random Access 
Memory), 116 
Random Access Memory (RAM), 
116 
Random bad pixels (shot noise), 205 
Random forest classifier, in machine 
learning, 444 
Random samples 
overview of, 132 
simple random sampling, 565-566 
stratified random sampling, 
565-566 
Range 
measures of dispersion, 138 
univariate statistics, 142 
Rapid access mass storage, 122-123 
RapidEye, Inc. systems, 
multispectral imaging using 
linear arrays, 93, 95 
Raster neighborhood analysis 
qualitative modeling, 292-293 
quantitative modeling, 293 
Raw images, 157 
Rayleigh scattering, 197-198 
RCD30 (Leica) medium-format 
camera, 102 
Read-Only Memory (ROM), 116 
Reasoning, inference and, 434 
Red-edge Position (REP) 
based on derivative ratio, 496 
determining, 332, 339 
indices for hyperspectral data 
analysis, 493-494 
Red-edge Vegetation Stress Index 
(RVSI), 494 
Red-Green-Blue (RGB) 
color coordinate system, 160 
color look-up tables, 161 
transformation to/from IHS, 
169-173 
Reduced Simple Ratio (RSR) 
for hyperspectral data 
analysis, 492 
vegetation indices, 331, 336-337 
Redundant arrays of inexpensive 
hard disks (RAID), 122-123 
References, change detection, 
551-555 
Reflectance of radiation 
atmospheric energy-matter 
interactions, 200-201 


factors controlling leaf 
reflectance, 315 
hemispherical reflectance, 202 
leaf additive reflectance, 321-323 
radiometric concepts, 201 
spectral reflectance curves, 203 
Refraction of light, atmospheric 
energy-matter interactions, 
196-197 
Region of interest (ROI) 
specifying in change detection 
process, 501-502 
supervised classification and, 362 
Regions, electromagnetic spectrum, 
190 
Regression, multiple-date image 
normalization for relative 
atmospheric correction, 224 
Regression kriging, for image 
fusion, 175 
Regression learning, 433 
Regression trees 
classification based on machine 
learning, 436-438 
in expert systems, 430 
machine learning and, 440 
Relative atmospheric correction 
of attenuation, 223-224 
multiple-date image 
normalization for, 224-230 
single-image normalization 
for, 224 
Relative radiometric correction, 212 
Remote sensing 
advantages/limitations, 7-8 
angular information and, 18-20 
as art, 7 
career opportunities, 29-30 
change detection considerations, 
504-506 
data analysis, 23-27 
data collection, 10-12 
defined, 3 
earth observation economics, 
28-29 
earth resource analysis 
perspective, 30-32 
expert systems applied to remote 
sensor data, 435 
information presentation, 27—28 
instruments, 4 
overview of, 3-4 
polarization information and, 
17-18 
process of, 8-9 
radiometric resolution and, 17 
references, 32—35 
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satellite systems, 20-23 
as science, 4—6 
spatial resolution and, 13-16 
spectral resolution and, 12-13 
statement of problem, 8-9 
suborbital (airborne) systems, 20 
temporal resolution and, 16-17 
vegetation indices in, 325 
REP. see Red-edge Position (REP) 
Resampling methods, intensity 
interpolation, 267 
ResourceSat, ISRO remote sensing 
systems, 85, 601 
Response design 
assessing accuracy of change 
detection map, 576 
obtaining ground reference 
information, 568-569 
RGB. see Red-Green-Blue (RGB) 
Roberts edge detectors 
nonlinear edge enhancement, 300 
for spatial filtering, 301 
ROL. see Region of interest (ROI) 
ROM (Read-Only Memory), 116 
RSR. see Reduced Simple Ratio 
(RSR) 
RuleQuest, machine learning 
and, 440 
Rules 
classification based on human- 
derived rules, 435-436 
from decision trees to production 
rules, 441-444 
in knowledge representation 
process, 432-433 
specifying in expert systems, 
433-434 
RVSI (Red-edge Vegetation Stress 
Index), 494 


Ss 
SAM (Spectral angle mapping) 
algorithms, 479-480 
Samples 
defined, 132 
sample size in assessing accuracy 
of derived thematic map, 
563-564 
Sampling error, 132 
Sampling frame, 558 
Sampling schemes 
assessing accuracy of change 
detection map, 576-577 
cluster sampling, 567 
overview of, 564—565 
simple random sampling, 
565-566 
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Sampling schemes (continued) 
stratified random sampling, 
566-568 
stratified systematic unaligned 
sampling, 567 
systematic sampling, 566 
Sampling theory, 132 
Sampling unit, 558 
SARVI (Soil and Atmospherically 
Resistant Vegetation Index), 
331, 335 
SASI-600 (Short-wavelength 
infrared-600), imaging 
spectrometry using linear and 
area arrays, 99, 600 
Satellite systems 
commercial remote sensing data 
providers, 23 
EROS, 92, 95, 600 
ERTS program, 48-51 
GOES, 64-67 
hyperspectral sensors, 460 
instantaneous-Field-Of-View 
(IFOV) and, 14 
KOMPSAT, 85-87, 601 
remote sensing, 20-23 
SeaStar, 71-73 
sentinel-2, 87-88, 602 
Terra, 88—90, 97 
Saturation, intensity-hue-saturation 
(IHS), 169-172 
SAVI (Soil Adjusted Vegetation 
Index), 331, 334-335 
Scanning mirrors, in multispectral 
imaging. see Multispectral 
imaging, using discrete 
detectors and scanning mirrors 
Scattering of radiation 
mie scattering, 198 
nonselective scattering, 198 
Rayleigh scattering, 197-198 
Scene modeling, data analysis 
and, 27 
SDTS (Spatial Data Transfer 
Standard), 130 
SeaStar satellites, 70—73 
Second-order statistics, in spatial 
domain, 345-348 
Selection rules, electrons, 193 
Self-organizing map (SOM), neural 
networks, 450-451 
Semivariance, calculating average, 
148-149 
Semivariogram. see also Empirical 
semivariogram 
model, 151 
overview of, 147-148 


texture statistics based on, 
349-350 
Sensors. see also Remote sensing 
advantages/limitations of remote 
sensing, 7-8 
overview of, 7 
passive and active, 10 
Sentinel-2 satellite system, 
multispectral imaging using 
linear arrays, 87-88, 602 
Serial processing, in digital image 
processing, 116 
SFIM (Smoothing Filter-based 
Intensity Modulation), for 
image fusion, 175 
Shape criterion, of image 
objects, 414 
Shape measurement, display 
(output), 181 
Sheffield Index (SI), color 
composites, 167 
Short-wavelength infrared-600 
(SASI-600), imaging 
spectrometry using linear and 
area arrays, 99, 600 
Shot noise (random bad pixels), 
205-206 
SI (Sheffield Index), color 
composites, 167 
Signal-to-noise ratio (SNR), 
assessing quality of 
hyperspectral data, 465-467 
Simple band substitution, for image 
fusion, 169 
Simple random sampling, 565-566 
Simple Ratio (SR), vegetation index, 
325, 330-332 
Single-image normalization, for 
relative atmospheric 
correction, 224 
Skewed distributions, 138, 141 
SLIM-6, multispectral imaging using 
linear arrays, 93-94 
Slope effects, atmospheric 
correction, 230 
SMA (Spectral Mixture 
Analysis), 413 
SMA (Spectral mixture analysis), 
480-484 
Small-format digital cameras, 101-102 
Smoothing Filter-based Intensity 
Modulation (SFIM), for image 
fusion, 175 
Snell’s law, 196 
SNR (Signal-to-noise ratio), 
assessing quality of 
hyperspectral data, 465-467 


Sobel edge detectors 
nonlinear edge enhancement, 
299-300 
for spatial filtering, 301 
Soft data. see Fuzzy data 
Software 
consideration related to image 
processing, 125-126 
costs, 129-130 
image processing functions, 126 
programs, 126-129 
Soil Adjusted Vegetation Index 
(SAVI), 331, 334-335 
Soil and Atmospherically Resistant 
Vegetation Index (SARVIJ), 
331, 335 
Soil moisture, change detection 
environmental/developmental 
considerations, 507—508 
SOM (self-organizing map), neural 
networks, 450-451 
SOM (Space Oblique Mercator) 
map projection, 255, 259 
Sorting, post-classification, 423 
Sound Navigation and Ranging 
(SONAR) 
intrusive impact on data 
collection, 8 
remote sensing instruments, 4 
Space Imaging, Inc, multispectral 
imaging using linear arrays, 
90-91 
Space Oblique Mercator (SOM) 
map projection, 255, 259 
Space shuttle photography 
analog cameras, 106-107 
digital cameras, 106 
Spatial Data Transfer Standard 
(SDTS), 130 
Spatial domain 
edge enhancement in, 298-302 
first-order statistics, 340-341 
second-order statistics in, 
345-348 
spatial filtering in, 298-302 
Spatial filtering 
edge enhancement in spatial 
domain, 298—302 
Fourier transform, 302-306 
in frequency domain, 306-308 
high-frequency spatial filtering, 
297-298 
low-frequency spatial filtering, 
294-297 
overview of, 293 
spatial convolution filtering, 
293-294 


Spatial profiles (transects), in image 
enhancement, 277-279 
Spatial resolution 
change detection considerations, 
505 
in remote sensing process, 13-16 
SPCS (State Plane Coordinate 
Systems), 263 
Spectral angle mapping (SAM) 
algorithms, 479-480 
Spectral change vector analysis, 
542-543 
Spectral classes 
information classes compared 
with, 365 
unsupervised classification 
and, 402 
Spectral Mixture Analysis 
(SMA), 413 
Spectral mixture analysis (SMA), 
480-484 
Spectral profiles, in image 
enhancement, 279-282 
Spectral reflectance curves, 203 
Spectral resolution 
change detection considerations, 
505-506 
in remote sensing process, 
12-13 
Spectrometry. see Imaging 
spectrometry 
Spectroradiometer, in-situ data 
collection, 2—3 
Specular reflection, 200-201, 203 
Spongy mesophyll cells 
middle infrared interactions with 
water in, 323-325 
near infrared interactions within, 
320-323 
in photosynthesis, 316 
SPOT systems 
overview of, 76-81, 602 
SPOT 1,2, and 3, 76-79, 602 
SPOT 4 and 5, 79-81, 602 
SPOT 6 and 7, 79-81, 602 
SR (Simple Ratio), vegetation index, 
325, 330-332 
SS (Sum of squares), covariance 
and, 142 
Standard deviation 
contrast stretch, 284-285 
measures of dispersion, 140 
univariate statistics, 142, 144 
State Plane Coordinate Systems 
(SPCS), 262 
Statement of problem, in remote 
sensing process, 8-9 


Statistical analysis, open-source 
software for, 129 
Statistical methods, in feature 
selection for image 
classification, 386—387 
Statistical thresholds, in binary 
change detection, 522 
Statistical-empirical correction 
atmospheric correction, 231 
radiometric correction, 231 
Stefan-Boltzman law, 188 
Stratified random samples, 132, 
566-568 
Stratified systematic unaligned 
sampling, 567 
Suborbital (airborne) systems 
remote sensing measurement 
and, 4 
types of remote sensing 
systems, 20 
Subpixel classification, 480-484 
Subtractive color theory, 160 
Sum of products (SP), covariance 
and, 142 
Suomi, NPP system, 69-70 
Super (mainframe) computers, 115 
Supervised classification 
algorithm selection for image 
classification, 393 
Bhattacharyya distance, 388 
Coastal Change Analysis 
Program, 370-371 
Correlation Matrix feature 
selection, 388, 392 
Cowardin System, 371-373 
defined, 362 
divergence as measure of 
statistical separability, 
387-392 
feature selection for image 
classification, 382 
graphic methods of feature 
selection, 382—386 
Jeffreys-Matusita distance, 388 
Land-Based Classification 
Standard, 365-366 
Land-Cover Classification 
System, 374-375 
land-use/land-cover classification 
schemes, 364—365 
Land-Use/Land-Cover 
Classification System for Use 
with Remote Sensor Data, 
366-367 
maximum likelihood 
classification algorithm, 
398-400 
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maximum likelihood classifica- 
tion with prior probability 
information, 402 
maximum likelihood classifica- 
tion without prior probability 
information, 400-401 
minimum distance to means 
classification algorithm, 
395-396 
multiple-date composite images 
in change detection, 523 
National Land Cover Database 
classification system, 367-370 
National Vegetation 
Classification Standard, 371 
nearest-neighbor classifiers, 
396-398 
observations about classification 
schemes, 375-376 
overview of, 362-364 
parallelepiped classification 
algorithm, 393-395 
PCA feature selection, 392—393 
selecting classification 
algorithm, 393 
statistical methods of feature 
selection, 386—387 
training site selection and 
statistics extraction, 376-382 
Support vector machines (SVMs) 
analysis of hyperspectral 
data, 491 
in machine learning, 444-445 
Surveying, mapping sciences, 5—6 
Symmetric thresholds, in binary 
change detection, 522 
Systematic samples, 132 
overview of, 565-566 
stratified systematic unaligned 
sampling, 567 


T 
Tagged interchange file format 
(TIFF), 157 
Target radiance, in absolute 
atmospheric correction, 
214-215 
TASI-600 (Thermal infrared-600), 
100, 600 
Tasseled Cap Transformation. see 
Kauth-Thomas Tasseled Cap 
Transformation, vegetation 
index 
Temporal resolution 
change detection considerations, 
504-505 
in remote sensing process, 16-17 
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Temporary video displays. see Video 
display, temporary 
Terra satellite 
ASTER system and, 88-89, 598 
MISR system and, 89-90 
MODIS imaging 
spectrometer, 97, 599 
Terrain energy-matter interactions 
hemispherical reflectance, 
absorptance, and transmit- 
tance, 202-203 
overview of, 201-202 
radiant flux density, 203-204 
Texture transformation 
conditional variance detection, 
342-343 
Edge-preserving Spectral- 
Smoothing variance texture, 
341-342 
first-order statistics in spatial 
domain, 340-341 
min-max texture operator, 
343-344 
Moran's / spatial autocorrelation 
metric, 344-345 
overview of, 340 
second-order statistics in spatial 
domain, 345-348 
texture statistics based on 
semivariogram, 349-350 
texture units as elements of 
texture spectrum, 
348-349 
Texture units, as elements of texture 
spectrum, 348-349 
Thematic attributes, specifying 
in change detection 
process, 501 
Thematic information extraction. 
see Pattern recognition 
Thematic map, accuracy of derived 
change detection, 575-578 
cluster sampling, 567 
descriptive evaluation of error 
matrices, 569-570 
discrete multivariate techniques 
applied to error matrices, 
570-571 
error matrix, 561-562 
fuzzification of error matrix, 
571-575 
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simple random sampling, 565-566 

sources of error, 558-561 

steps in, 557-558 

stratified random sampling, 
566-568 

stratified systematic unaligned 
sampling, 567 

systematic sampling, 566 
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information, 562—563 
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considerations, 510—512 


TIFF (Tagged interchange file 


format), 157 


Time period, specifying in change 


detection process, 502 
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artificial neural networks, 
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decision tree training, 441 

machine learning and, 440 
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selection and statistics extraction, 
376-382 
supervised classification, 362 
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Index (TCARJ), 494 
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absolute atmospheric correction, 
215-216 
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radiometric concepts, 201 

terrain energy-matter 
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TVI (Triangular Vegetation Index), 
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UAVs (Unmanned Aerial Vehicles), 


remote sensing instruments 
carried by, 4 
ItraCam Eagle large-format 
camera, 104-105 
Itraspectral remote sensing, 12, 44 
niform distribution, in 
sampling, 133 
nited States Global Change 
Research Program 
(USGCRP), 20 
nivariate statistics 
collecting training data, 379-382 
example, 144 
measure of central tendency, 138 
measure of dispersion, 138-141 
measures of distribution, 141 
overview of, 138 
niversal Transverse Mercator 
(UTM) map projection, 
255, 257-259 
nmanned Aerial Vehicles (UAVs), 
remote sensing instruments 
carried by, 4 
nsupervised classification 
chain method, 402-406 
defined, 362 
ISODATA method, 406-411 
multiple-date composite image in 
change detection, 523 
overview of, 402 
unsupervised cluster busting, 412 
rban-suburban phenological 
cycles, 509-510 
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Fish & Wildlife Service 
Classification of Wetlands and 
Deepwater Habitats of the 
United States, 371-373, 595 

Land-Use/Land-Cover Classifica- 
tion System for Use with 
Remote Sensor Data, 366—367 

National Vegetation 
Classification Standard, 371 


geostatistical analysis in, 
578-579 

image metadata and, 579 

lineage of remote sensing-derived 
products, 579-580 

overview of, 557 

references, 580-582 


Transpiration, water loss in 
plants, 324 

Triangular Vegetation Index (TVI), 
331, 336 

Trichromatic coefficients, 173 

Turgid/turgidity, in water transport 
in plants, 323 


space shuttle photography, 
106-107 
User interface, expert systems, 430 
User’s accuracy, 569 
USGCRP (United States Global 
Change Research 
Program), 20 


USGS (U.S. Geological Survey), 
366-367 

Utilities, for image processing, 126 

UTM (Universal Transverse 
Mercator) map projection, 
255, 257-259 


Vv 
VANUI (Vegetation Adjusted 
Nighttime Light Urban Index), 
332, 338-339 
VARI (Visible Atmospherically 
Resistant Index), 331, 338 
Variables, in classification based on 
human-derived rules, 436 
Variance 
calculating average 
semivariance, 148 
covariance, 142 
of samples, 140 
univariate statistics, 142, 144 
Variance-Covariance matrix, 
379-382 
Variography, 147-149 
Vegetation, natural and cultural, 371 
Vegetation Adjusted Nighttime 
Light Urban Index (VANUD, 
332, 338-339 
Vegetation indices 
Aerosol Free Vegetation Index, 
335-336 
Atmospherically Resistant 
Vegetation Index, 335 
Cellulose Absorption Index, 339 
Enhanced Vegetation Index, 336 
factors controlling leaf 
reflectance, 315 
interaction of light with pigments 
in palisade mesophyll cells, 
316-320 
Kauth-Thomas Tasseled Cap 
Transformation, 327, 329 


Leaf Water Content Index, 
333-334 

MERIS Terrestrial Chlorophyll 
Index, 339-340 

middle infrared interaction with 
water in spongy mesophyll 
cells, 323-325 

near infrared interaction within 
spongy mesophyll cells, 320— 
323 

Normalized Burn Ratio, 340 

Normalized Difference Built-up 
Index, 338 

Normalized Difference Moisture 
or Water Index, 332-333 

Normalized Difference Vegeta- 
tion Index, 325-327, 329 

overview of, 314-315 

Perpendicular Vegetation 
Index, 333 

Photochemical Reflectance 
Index, 339 

Red-edge Position 
Determination, 339 

Reduced Simple Ratio, 336-337 

remote sensing using derived 
vegetation indices, 325, 
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indices, 325 

Soil Adjusted Vegetation Index, 
334-335 

Soil and Atmospherically Resis- 
tant Vegetation Index, 335 
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Reflectance Index, 337—338 

Triangular Vegetation Index, 336 

Vegetation Adjusted Nighttime 
Light Urban Index, 338-339 

vegetation suppression, 340 

Visible Atmospherically Resistant 
Index, 338 
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Vegetation phenology, 508-509 
Vegetation suppression, 340 
Video digitization, 39 
Video display, temporary 
bitmapped graphics, 157-160 
black-and-white and color 
brightness maps, 157 
color composites, 164-167 
color look-up tables, 
160-164 
overview of, 156 
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system, 160 
Visible Atmospherically Resistant 
Index (VARI), 
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Visual on-screen change detection 
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Ww 
Wave model, of electromagnetic 
energy, 187-191 
Wavelength 
electromagnetic radiation, 
187-189 
solar radiation, 199 
Web-enabled Landsat Data 
(WELD) project, 270 
Wien’s displacement law, 188 
Workstation computers, 115 
WorldView-1,2, and 3, 
multispectral imaging using 
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hyperspectral data analysis, 
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INTRODUCTORY 
DIGITAL IMAGE PROCESSING 


A Remote Sensing Perspective 


Ath Edition 
John R. Jensen 


Digital image processing of remotely sensed data is a rapidly developing technology 
used by scientists and agencies throughout the world to obtain information for managing 
natural resources and many other applications. Introductory Digital Image Processing: 
A Remote Sensing Perspective, 4th Edition introduces how remote sensing imagery 
is collected. It then presents the algorithms used to radiometrically and geometrically 
prepare digital imagery for further analysis. Great emphasis is then placed on the use 
of state-of-the-art digital image processing techniques to enhance single-date images 
and to extract useful thematic information from the imagery (e.g., land cover). The book 
then reviews algorithms for digital change detection using multiple dates of imagery. 
Change information is used to identify manmade and physical processes at work to 
solve real-world problems. The final chapter describes how to assess the accuracy of 
thematic map products derived from remote sensing. 


About the Cover: The top image is an artist’s rendition of the Landsat 8 Earth observation satellite in orbit 
over the southeastern United States. Since 1972, the Landsat satellites have provided a continuous record 
of Earth's changing land surfaces. It is the world’s largest and most sustained collection of Earth observation 
imagery. The lower left image is a Landsat 8 scene of the Cambridge Gulf in Northwestern Australia. 
Geoscience Australia enhanced the image to display sediment and nutrient patterns in the tropical estuary 
area, and the complex patterns and conditions in the vegetated areas. The lower right image is a natural 
color Landsat 8 image of the salt evaporation ponds in the southern part of San Francisco Bay, California. 
The image was processed to highlight the ponds, which are designed to produce salts from sea water. Salt 
ponds range in color from blue green to deep magenta, where the color indicates the salinity of the ponds 
and the type of microorganisms breeding in it. Introductory Digital Image Processing explores the tools and 
techniques used to create and analyze images such as these. 


GEOGRAPHIC 


INFORMATION SCIENCE 


Please visit us at www.pearsonhighered.com for more information. To order any 
of our products, contact our customer service department at (800) 824-7799, 


(201) 767-5021 outside of the U.S., or visit your campus bookstore. | | | | | | | 
9 "780134105 


www.pearsonhighered.com 


Bet if 978- ee ine pears : 


PEARSON ALWAYS LEARNING 


